[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:04:31 dist-test-slave-zj8q krb5kdc[1643](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:04:31 dist-test-slave-zj8q krb5kdc[1643](info): set up 2 sockets
Aug 08 06:04:31 dist-test-slave-zj8q krb5kdc[1643](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 06:04:32.149092 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.007s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:04:32 dist-test-slave-zj8q krb5kdc[1643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:04:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:04:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.151.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
I20260808 06:04:32.269835 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:46039
--webserver_interface=127.1.151.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:38339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:46039
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:04:32.379398 1659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:32.379637 1659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:32.379675 1659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:32.383529 1659 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:04:32.383587 1659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:32.383605 1659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:32.383620 1659 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:04:32.383632 1659 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:04:32.388674 1659 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:38339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:46039
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:46039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:32.389729 1659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:32.390805 1659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:32.399533 1669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:32.399546 1659 server_base.cc:1034] running on GCE node
W20260808 06:04:32.399492 1667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:32.399471 1672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:32.400272 1659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:32.401337 1659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:32.402503 1659 hybrid_clock.cc:648] HybridClock initialized: now 1786169072402500 us; error 67 us; skew 500 ppm
Aug 08 06:04:32 dist-test-slave-zj8q krb5kdc[1643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169072, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:04:32.405211 1659 init.cc:377] Logged in from keytab as kudu/127.1.151.190@KRBTEST.COM (short username kudu)
I20260808 06:04:32.406045 1659 webserver.cc:460] Webserver started at http://127.1.151.190:34271/ using document root <none> and password file <none>
I20260808 06:04:32.406602 1659 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:32.406685 1659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:32.406880 1659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:32.408876 1659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "e6281a42deb149c4996faa273982d1dd"
format_stamp: "Formatted at 2026-08-08 06:04:32 on dist-test-slave-zj8q"
server_key: "7e63c09d0c3c013b2976f84634891bf3"
server_key_iv: "bb07109249a98ddbfe665a5abd1160af"
server_key_version: "encryptionkey@0"
I20260808 06:04:32.409405 1659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "e6281a42deb149c4996faa273982d1dd"
format_stamp: "Formatted at 2026-08-08 06:04:32 on dist-test-slave-zj8q"
server_key: "7e63c09d0c3c013b2976f84634891bf3"
server_key_iv: "bb07109249a98ddbfe665a5abd1160af"
server_key_version: "encryptionkey@0"
I20260808 06:04:32.412957 1659 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260808 06:04:32.415282 1683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:32.416430 1659 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 06:04:32.416531 1659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "e6281a42deb149c4996faa273982d1dd"
format_stamp: "Formatted at 2026-08-08 06:04:32 on dist-test-slave-zj8q"
server_key: "7e63c09d0c3c013b2976f84634891bf3"
server_key_iv: "bb07109249a98ddbfe665a5abd1160af"
server_key_version: "encryptionkey@0"
I20260808 06:04:32.416637 1659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:32.441538 1659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:32.444473 1659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:32.444645 1659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:32.451509 1659 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:46039
I20260808 06:04:32.451527 1765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:46039 every 8 connection(s)
I20260808 06:04:32.452399 1659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:04:32.455108 1766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:32.456120 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 1659
I20260808 06:04:32.456240 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:04:32.457506 1630 external_mini_cluster.cc:1428] Setting key 5449eab726162b11035cd26c1ea331d9
I20260808 06:04:32.460736 1766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6281a42deb149c4996faa273982d1dd: Bootstrap starting.
I20260808 06:04:32.463460 1766 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6281a42deb149c4996faa273982d1dd: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:32.464341 1766 log.cc:824] T 00000000000000000000000000000000 P e6281a42deb149c4996faa273982d1dd: Log is configured to *not* fsync() on all Append() calls
I20260808 06:04:32.466406 1766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6281a42deb149c4996faa273982d1dd: No bootstrap required, opened a new log
I20260808 06:04:32.470819 1766 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6281a42deb149c4996faa273982d1dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6281a42deb149c4996faa273982d1dd" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 46039 } }
I20260808 06:04:32.471091 1766 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6281a42deb149c4996faa273982d1dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:32.471148 1766 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6281a42deb149c4996faa273982d1dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6281a42deb149c4996faa273982d1dd, State: Initialized, Role: FOLLOWER
I20260808 06:04:32.471606 1766 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6281a42deb149c4996faa273982d1dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6281a42deb149c4996faa273982d1dd" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 46039 } }
I20260808 06:04:32.471771 1766 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6281a42deb149c4996faa273982d1dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:04:32.471836 1766 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6281a42deb149c4996faa273982d1dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:04:32.471925 1766 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6281a42deb149c4996faa273982d1dd [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:32.472797 1766 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6281a42deb149c4996faa273982d1dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6281a42deb149c4996faa273982d1dd" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 46039 } }
I20260808 06:04:32.473155 1766 leader_election.cc:302] T 00000000000000000000000000000000 P e6281a42deb149c4996faa273982d1dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6281a42deb149c4996faa273982d1dd; no voters:
I20260808 06:04:32.473490 1766 leader_election.cc:288] T 00000000000000000000000000000000 P e6281a42deb149c4996faa273982d1dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:04:32.473572 1775 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6281a42deb149c4996faa273982d1dd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:04:32.473816 1775 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6281a42deb149c4996faa273982d1dd [term 1 LEADER]: Becoming Leader. State: Replica: e6281a42deb149c4996faa273982d1dd, State: Running, Role: LEADER
I20260808 06:04:32.474124 1775 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6281a42deb149c4996faa273982d1dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6281a42deb149c4996faa273982d1dd" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 46039 } }
I20260808 06:04:32.474828 1766 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6281a42deb149c4996faa273982d1dd [sys.catalog]: configured and running, proceeding with master startup.
Aug 08 06:04:32 dist-test-slave-zj8q krb5kdc[1643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:04:32.477219 1776 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6281a42deb149c4996faa273982d1dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6281a42deb149c4996faa273982d1dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6281a42deb149c4996faa273982d1dd" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 46039 } } }
I20260808 06:04:32.477825 1776 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6281a42deb149c4996faa273982d1dd [sys.catalog]: This master's current role is: LEADER
I20260808 06:04:32.477437 1778 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6281a42deb149c4996faa273982d1dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6281a42deb149c4996faa273982d1dd. Latest consensus state: current_term: 1 leader_uuid: "e6281a42deb149c4996faa273982d1dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6281a42deb149c4996faa273982d1dd" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 46039 } } }
I20260808 06:04:32.478027 1778 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6281a42deb149c4996faa273982d1dd [sys.catalog]: This master's current role is: LEADER
W20260808 06:04:32.479894 1796 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e6281a42deb149c4996faa273982d1dd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 06:04:32.479976 1796 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 06:04:32.480053 1801 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:04:32.480909 1801 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:04:32.483254 1769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:32.460665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56762 (local address 127.1.151.190:46039)
0808 06:04:32.461118 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:32.461138 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:32.462211 (+ 1073us) server_negotiation.cc:407] Connection header received
0808 06:04:32.466181 (+ 3970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:32.466200 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:32.466491 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:32.466763 (+ 272us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:32.469589 (+ 2826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:32.470619 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:32.472305 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:32.472746 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:32.472858 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:32.477797 (+ 4939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:32.477804 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:32.477818 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:32.477848 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:32.479625 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:32.480175 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:32.480180 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:32.480184 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:32.480290 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:32.480611 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:32.480614 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:32.480616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:32.481076 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:32.481262 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:32.481802 (+ 540us) server_negotiation.cc:299] Negotiation successful
0808 06:04:32.482048 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":135,"threads_started":1}
I20260808 06:04:32.486940 1801 catalog_manager.cc:1347] Generated new cluster ID: 4f7fdb4b2cfc4689ba2573e24f3dc64f
I20260808 06:04:32.486992 1801 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:04:32.506621 1801 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:04:32.507434 1801 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:04:32.515383 1801 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6281a42deb149c4996faa273982d1dd: Generated new TSK 0
I20260808 06:04:32.516018 1801 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.151.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab.
Entry for principal kudu/127.1.151.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab.
I20260808 06:04:32.548926 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.129:0
--local_ip_for_outbound_sockets=127.1.151.129
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:46039
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:38339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:04:32.659673 1810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:32.659909 1810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:32.659950 1810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:32.663815 1810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:32.663861 1810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:32.663914 1810 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.129
I20260808 06:04:32.668892 1810 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:38339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:46039
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1810
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.129
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:32.669756 1810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:32.670696 1810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:32.679697 1825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:32.679730 1819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:32.679730 1821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:32.679925 1810 server_base.cc:1034] running on GCE node
I20260808 06:04:32.680402 1810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:32.680979 1810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:32.682202 1810 hybrid_clock.cc:648] HybridClock initialized: now 1786169072682181 us; error 34 us; skew 500 ppm
Aug 08 06:04:32 dist-test-slave-zj8q krb5kdc[1643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169072, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:04:32.684902 1810 init.cc:377] Logged in from keytab as kudu/127.1.151.129@KRBTEST.COM (short username kudu)
I20260808 06:04:32.685781 1810 webserver.cc:460] Webserver started at http://127.1.151.129:38693/ using document root <none> and password file <none>
I20260808 06:04:32.686358 1810 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:32.686429 1810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:32.686627 1810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:32.688565 1810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-0/data/instance:
uuid: "577ca4bbd247473891b36080ecbba66d"
format_stamp: "Formatted at 2026-08-08 06:04:32 on dist-test-slave-zj8q"
server_key: "1f5abd5fd3857247038d68f37b08a5cc"
server_key_iv: "aa6b57b1c5f58d0a92f634030c3d708c"
server_key_version: "encryptionkey@0"
I20260808 06:04:32.689102 1810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance:
uuid: "577ca4bbd247473891b36080ecbba66d"
format_stamp: "Formatted at 2026-08-08 06:04:32 on dist-test-slave-zj8q"
server_key: "1f5abd5fd3857247038d68f37b08a5cc"
server_key_iv: "aa6b57b1c5f58d0a92f634030c3d708c"
server_key_version: "encryptionkey@0"
I20260808 06:04:32.692643 1810 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 06:04:32.694991 1835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:32.696290 1810 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:04:32.696419 1810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-0/wal
uuid: "577ca4bbd247473891b36080ecbba66d"
format_stamp: "Formatted at 2026-08-08 06:04:32 on dist-test-slave-zj8q"
server_key: "1f5abd5fd3857247038d68f37b08a5cc"
server_key_iv: "aa6b57b1c5f58d0a92f634030c3d708c"
server_key_version: "encryptionkey@0"
I20260808 06:04:32.696532 1810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:32.707325 1810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:32.709857 1810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:32.710000 1810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:32.710534 1810 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:04:32.711275 1810 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:04:32.711323 1810 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:32.711357 1810 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:04:32.711377 1810 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:32.720427 1810 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.129:41637
I20260808 06:04:32.720453 1999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.129:41637 every 8 connection(s)
I20260808 06:04:32.721282 1810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
I20260808 06:04:32.724745 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 1810
I20260808 06:04:32.724833 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance
I20260808 06:04:32.725056 1630 external_mini_cluster.cc:1428] Setting key 35709775f9af586d29a742d951228fe6
Aug 08 06:04:32 dist-test-slave-zj8q krb5kdc[1643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169072, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:04:32.734272 1769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:32.722959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:49607 (local address 127.1.151.190:46039)
0808 06:04:32.723090 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:32.723094 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:32.723895 (+ 801us) server_negotiation.cc:407] Connection header received
0808 06:04:32.725272 (+ 1377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:32.725276 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:32.725335 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:32.725402 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:32.726695 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:32.727219 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:32.728333 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:32.728556 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:32.728649 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:32.730993 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:32.730996 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:32.730998 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:32.731022 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:32.732305 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:32.732785 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:32.732787 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:32.732789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:32.732866 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:32.733180 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:32.733183 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:32.733185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:32.733360 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:32.733447 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:32.733941 (+ 494us) server_negotiation.cc:299] Negotiation successful
0808 06:04:32.734064 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
WARNING: no policy specified for kudu/127.1.151.130@KRBTEST.COM; defaulting to no policy
I20260808 06:04:32.735033 2005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:32.723316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:46039 (local address 127.1.151.129:49607)
0808 06:04:32.723760 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:32.723800 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:32.725054 (+ 1254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:32.725587 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:32.725594 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:32.726022 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:32.726558 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:32.726571 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:32.727337 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:32.727340 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:32.728214 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:32.728224 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:32.728704 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:32.728708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:32.728867 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:32.729381 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:04:32.729396 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:32.730840 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:32.732414 (+ 1574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:32.732420 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:32.732434 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:32.732690 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:32.732948 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:32.732951 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:32.732953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:32.733086 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:32.733461 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:32.733467 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:32.733714 (+ 247us) client_negotiation.cc:769] Sending connection context
0808 06:04:32.733878 (+ 164us) client_negotiation.cc:240] Negotiation successful
0808 06:04:32.734220 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":100,"threads_started":1}
I20260808 06:04:32.736248 2001 heartbeater.cc:347] Connected to a master server at 127.1.151.190:46039
I20260808 06:04:32.736459 2001 heartbeater.cc:464] Registering TS with master...
I20260808 06:04:32.736927 2001 heartbeater.cc:511] Master 127.1.151.190:46039 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.130@KRBTEST.COM" created.
I20260808 06:04:32.738646 1721 ts_manager.cc:194] Registered new tserver with Master: 577ca4bbd247473891b36080ecbba66d (127.1.151.129:41637)
I20260808 06:04:32.739897 1721 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.129@KRBTEST.COM'} at 127.1.151.129:49607
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab.
Entry for principal kudu/127.1.151.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab.
I20260808 06:04:32.752480 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.130:0
--local_ip_for_outbound_sockets=127.1.151.130
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:46039
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:38339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:04:32.862807 2008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:32.863031 2008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:32.863071 2008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:32.866912 2008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:32.866962 2008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:32.867017 2008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.130
I20260808 06:04:32.872093 2008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:38339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:46039
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.130
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:32.873039 2008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:32.873941 2008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:32.883198 2016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:32.883271 2018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:32.883203 2015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:32.883545 2008 server_base.cc:1034] running on GCE node
I20260808 06:04:32.883940 2008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:32.884524 2008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:32.885696 2008 hybrid_clock.cc:648] HybridClock initialized: now 1786169072885677 us; error 29 us; skew 500 ppm
Aug 08 06:04:32 dist-test-slave-zj8q krb5kdc[1643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169072, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:04:32.888265 2008 init.cc:377] Logged in from keytab as kudu/127.1.151.130@KRBTEST.COM (short username kudu)
I20260808 06:04:32.889063 2008 webserver.cc:460] Webserver started at http://127.1.151.130:37835/ using document root <none> and password file <none>
I20260808 06:04:32.889575 2008 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:32.889649 2008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:32.889854 2008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:32.891816 2008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-1/data/instance:
uuid: "8e43dfd3eecc47a09e5f7c2c3bab96e2"
format_stamp: "Formatted at 2026-08-08 06:04:32 on dist-test-slave-zj8q"
server_key: "78778b89f44a5ea8e5bd857532098c60"
server_key_iv: "9ae083a17324d9b4e0e8cacc1921bf47"
server_key_version: "encryptionkey@0"
I20260808 06:04:32.892333 2008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance:
uuid: "8e43dfd3eecc47a09e5f7c2c3bab96e2"
format_stamp: "Formatted at 2026-08-08 06:04:32 on dist-test-slave-zj8q"
server_key: "78778b89f44a5ea8e5bd857532098c60"
server_key_iv: "9ae083a17324d9b4e0e8cacc1921bf47"
server_key_version: "encryptionkey@0"
I20260808 06:04:32.895895 2008 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 06:04:32.898078 2026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:32.899247 2008 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 06:04:32.899353 2008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-1/wal
uuid: "8e43dfd3eecc47a09e5f7c2c3bab96e2"
format_stamp: "Formatted at 2026-08-08 06:04:32 on dist-test-slave-zj8q"
server_key: "78778b89f44a5ea8e5bd857532098c60"
server_key_iv: "9ae083a17324d9b4e0e8cacc1921bf47"
server_key_version: "encryptionkey@0"
I20260808 06:04:32.899438 2008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:32.925822 2008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:32.928936 2008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:32.929111 2008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:32.929679 2008 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:04:32.930621 2008 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:04:32.930678 2008 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:32.930733 2008 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:04:32.930757 2008 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:32.939308 2008 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.130:40343
I20260808 06:04:32.939335 2202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.130:40343 every 8 connection(s)
I20260808 06:04:32.940151 2008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
Aug 08 06:04:32 dist-test-slave-zj8q krb5kdc[1643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169072, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:04:32.949046 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 2008
I20260808 06:04:32.949158 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance
I20260808 06:04:32.949450 1630 external_mini_cluster.cc:1428] Setting key 525da1a3de607482cf97af5f1823a64a
I20260808 06:04:32.953270 1769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:32.941963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:60215 (local address 127.1.151.190:46039)
0808 06:04:32.942102 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:32.942106 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:32.942972 (+ 866us) server_negotiation.cc:407] Connection header received
0808 06:04:32.944247 (+ 1275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:32.944249 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:32.944296 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:32.944366 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:32.945461 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:32.945993 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:32.947082 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:32.947317 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:32.947394 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:32.949736 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:32.949740 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:32.949742 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:32.949765 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:32.951219 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:32.951790 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:32.951793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:32.951794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:32.951858 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:32.952178 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:32.952180 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:32.952181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:32.952338 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:32.952403 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:32.952982 (+ 579us) server_negotiation.cc:299] Negotiation successful
0808 06:04:32.953116 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 06:04:32.954062 2212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:32.942392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:46039 (local address 127.1.151.130:60215)
0808 06:04:32.942831 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:32.942870 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:32.944056 (+ 1186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:32.944512 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:32.944518 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:32.944884 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:32.945348 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:32.945361 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:32.946089 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:32.946091 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:32.946973 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:32.946982 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:32.947426 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:32.947429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:32.947559 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:32.948028 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:04:32.948043 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:32.949558 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:32.951335 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:32.951341 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:32.951355 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:32.951707 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:32.951930 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:32.951933 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:32.951935 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:32.952103 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:32.952421 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:32.952427 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:32.952732 (+ 305us) client_negotiation.cc:769] Sending connection context
0808 06:04:32.952929 (+ 197us) client_negotiation.cc:240] Negotiation successful
0808 06:04:32.953244 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":116,"threads_started":1}
I20260808 06:04:32.955148 2206 heartbeater.cc:347] Connected to a master server at 127.1.151.190:46039
I20260808 06:04:32.955389 2206 heartbeater.cc:464] Registering TS with master...
I20260808 06:04:32.955937 2206 heartbeater.cc:511] Master 127.1.151.190:46039 requested a full tablet report, sending...
I20260808 06:04:32.957037 1721 ts_manager.cc:194] Registered new tserver with Master: 8e43dfd3eecc47a09e5f7c2c3bab96e2 (127.1.151.130:40343)
I20260808 06:04:32.957583 1721 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.130@KRBTEST.COM'} at 127.1.151.130:60215
WARNING: no policy specified for kudu/127.1.151.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab.
Entry for principal kudu/127.1.151.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab.
I20260808 06:04:32.977453 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.131:0
--local_ip_for_outbound_sockets=127.1.151.131
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:46039
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:38339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:04:33.086901 2216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:33.087144 2216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:33.087203 2216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:33.091156 2216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:33.091210 2216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:33.091292 2216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.131
I20260808 06:04:33.096449 2216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:38339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:46039
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.131
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:33.097470 2216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:33.098467 2216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:33.107960 2226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:33.107937 2233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:33.107930 2228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:33.108318 2216 server_base.cc:1034] running on GCE node
I20260808 06:04:33.108676 2216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:33.109278 2216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:33.110445 2216 hybrid_clock.cc:648] HybridClock initialized: now 1786169073110429 us; error 28 us; skew 500 ppm
Aug 08 06:04:33 dist-test-slave-zj8q krb5kdc[1643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169073, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:04:33.113070 2216 init.cc:377] Logged in from keytab as kudu/127.1.151.131@KRBTEST.COM (short username kudu)
I20260808 06:04:33.113946 2216 webserver.cc:460] Webserver started at http://127.1.151.131:34863/ using document root <none> and password file <none>
I20260808 06:04:33.114502 2216 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:33.114563 2216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:33.114742 2216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:33.116672 2216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-2/data/instance:
uuid: "73fe52abeac849f49dd2507d9cf2a841"
format_stamp: "Formatted at 2026-08-08 06:04:33 on dist-test-slave-zj8q"
server_key: "f691309039c9611304e236439005569d"
server_key_iv: "9d1b981f22a6c5b4182175e59ee04b68"
server_key_version: "encryptionkey@0"
I20260808 06:04:33.117175 2216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance:
uuid: "73fe52abeac849f49dd2507d9cf2a841"
format_stamp: "Formatted at 2026-08-08 06:04:33 on dist-test-slave-zj8q"
server_key: "f691309039c9611304e236439005569d"
server_key_iv: "9d1b981f22a6c5b4182175e59ee04b68"
server_key_version: "encryptionkey@0"
I20260808 06:04:33.120803 2216 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 06:04:33.123109 2244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:33.124269 2216 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:04:33.124361 2216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-2/wal
uuid: "73fe52abeac849f49dd2507d9cf2a841"
format_stamp: "Formatted at 2026-08-08 06:04:33 on dist-test-slave-zj8q"
server_key: "f691309039c9611304e236439005569d"
server_key_iv: "9d1b981f22a6c5b4182175e59ee04b68"
server_key_version: "encryptionkey@0"
I20260808 06:04:33.124439 2216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:33.142654 2216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:33.145299 2216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:33.145445 2216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:33.145977 2216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:04:33.146797 2216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:04:33.146843 2216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:33.146881 2216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:04:33.146903 2216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:33.155643 2216 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.131:36615
I20260808 06:04:33.155694 2433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.131:36615 every 8 connection(s)
I20260808 06:04:33.156471 2216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
I20260808 06:04:33.163188 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 2216
I20260808 06:04:33.163276 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance
I20260808 06:04:33.163511 1630 external_mini_cluster.cc:1428] Setting key dcbb1aba13e34b392ec81c69ba2f7cb7
Aug 08 06:04:33 dist-test-slave-zj8q krb5kdc[1643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169073, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:04:33.170377 1769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:33.158451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:41597 (local address 127.1.151.190:46039)
0808 06:04:33.158595 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:33.158600 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:33.159707 (+ 1107us) server_negotiation.cc:407] Connection header received
0808 06:04:33.161107 (+ 1400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:33.161111 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:33.161169 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:33.161251 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:33.162416 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:33.162949 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:33.164232 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:33.164534 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:33.164640 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:33.167130 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:33.167134 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:33.167137 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:33.167162 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:33.168485 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:33.168959 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:33.168962 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:33.168963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:33.169037 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:33.169345 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:33.169349 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:33.169350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:33.169517 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:33.169601 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:33.170069 (+ 468us) server_negotiation.cc:299] Negotiation successful
0808 06:04:33.170222 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 06:04:33.171077 2439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:33.159067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:46039 (local address 127.1.151.131:41597)
0808 06:04:33.159562 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:33.159602 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:33.160904 (+ 1302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:33.161431 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:33.161438 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:33.161769 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:33.162280 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:33.162296 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:33.163067 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:33.163071 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:33.164107 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:33.164119 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:33.164697 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:33.164702 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:33.164899 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:33.165447 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:04:33.165462 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:33.166981 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:33.168593 (+ 1612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:33.168599 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:33.168613 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:33.168866 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:33.169120 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:33.169123 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:33.169125 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:33.169257 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:33.169616 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:33.169622 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:33.169853 (+ 231us) client_negotiation.cc:769] Sending connection context
0808 06:04:33.170009 (+ 156us) client_negotiation.cc:240] Negotiation successful
0808 06:04:33.170334 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":135,"threads_started":1}
I20260808 06:04:33.172021 2434 heartbeater.cc:347] Connected to a master server at 127.1.151.190:46039
I20260808 06:04:33.172235 2434 heartbeater.cc:464] Registering TS with master...
I20260808 06:04:33.172685 2434 heartbeater.cc:511] Master 127.1.151.190:46039 requested a full tablet report, sending...
I20260808 06:04:33.173633 1721 ts_manager.cc:194] Registered new tserver with Master: 73fe52abeac849f49dd2507d9cf2a841 (127.1.151.131:36615)
I20260808 06:04:33.174177 1721 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.131@KRBTEST.COM'} at 127.1.151.131:41597
I20260808 06:04:33.177716 1630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 06:04:33 dist-test-slave-zj8q krb5kdc[1643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169073, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 06:04:33 dist-test-slave-zj8q krb5kdc[1643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169073, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.151.129@KRBTEST.COM
I20260808 06:04:33.204496 2447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:33.195178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46212 (local address 127.1.151.129:41637)
0808 06:04:33.195473 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:33.195486 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:33.195511 (+ 25us) server_negotiation.cc:407] Connection header received
0808 06:04:33.195611 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:33.195616 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:33.195803 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:33.195918 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:33.196523 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:33.197359 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:33.198201 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:33.198564 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:33.198651 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:33.201006 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:33.201015 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:33.201027 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:33.201059 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:33.202795 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:33.203259 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:33.203265 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:33.203266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:33.203372 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:33.203630 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:33.203633 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:33.203634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:33.203879 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:33.204049 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:33.204228 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 06:04:33.204355 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":87,"threads_started":1}
W20260808 06:04:33.205492 1880 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46212
Aug 08 06:04:33 dist-test-slave-zj8q krb5kdc[1643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 06:04:33 dist-test-slave-zj8q krb5kdc[1643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.129@KRBTEST.COM
I20260808 06:04:33.231243 2447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:33.223296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46218 (local address 127.1.151.129:41637)
0808 06:04:33.223429 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:33.223433 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:33.223564 (+ 131us) server_negotiation.cc:407] Connection header received
0808 06:04:33.223704 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:33.223708 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:33.223752 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:33.223816 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:33.224463 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:33.224993 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:33.225818 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:33.226060 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:33.226148 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:33.228356 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:33.228359 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:33.228361 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:33.228384 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:33.229744 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:33.230110 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:33.230114 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:33.230116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:33.230223 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:33.230461 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:33.230464 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:33.230466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:33.230648 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:33.230778 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:33.230984 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 06:04:33.231094 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 06:04:33.232120 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 1810
I20260808 06:04:33.245728 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 2008
I20260808 06:04:33.260555 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 2216
I20260808 06:04:33.274066 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 1659
2026-08-08T06:04:33Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2322 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:04:33 dist-test-slave-zj8q krb5kdc[2459](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:04:33 dist-test-slave-zj8q krb5kdc[2459](info): set up 2 sockets
Aug 08 06:04:33 dist-test-slave-zj8q krb5kdc[2459](info): commencing operation
krb5kdc: starting...
W20260808 06:04:34.427431 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:04:34 dist-test-slave-zj8q krb5kdc[2459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:04:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:04:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.151.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
I20260808 06:04:34.544983 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:45283
--webserver_interface=127.1.151.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:43235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:45283
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:04:34.655473 2492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:34.655736 2492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:34.655791 2492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:34.659657 2492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:04:34.659708 2492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:34.659734 2492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:34.659754 2492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:04:34.659780 2492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:04:34.665210 2492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:43235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:45283
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:45283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:34.666216 2492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:34.667173 2492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:34.675406 2502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:34.675379 2503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:34.675420 2505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:34.675535 2492 server_base.cc:1034] running on GCE node
I20260808 06:04:34.676244 2492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:34.677317 2492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:34.678479 2492 hybrid_clock.cc:648] HybridClock initialized: now 1786169074678433 us; error 57 us; skew 500 ppm
Aug 08 06:04:34 dist-test-slave-zj8q krb5kdc[2459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169074, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:04:34.681111 2492 init.cc:377] Logged in from keytab as kudu/127.1.151.190@KRBTEST.COM (short username kudu)
I20260808 06:04:34.681936 2492 webserver.cc:460] Webserver started at http://127.1.151.190:45669/ using document root <none> and password file <none>
I20260808 06:04:34.682531 2492 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:34.682602 2492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:34.682811 2492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:34.684824 2492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "848ae95361094fad8d0cbe776a8403c7"
format_stamp: "Formatted at 2026-08-08 06:04:34 on dist-test-slave-zj8q"
server_key: "4130e9839883e795c1424cf10a682f58"
server_key_iv: "e7d7a1ab61f4043c46d055a61c537c28"
server_key_version: "encryptionkey@0"
I20260808 06:04:34.685384 2492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "848ae95361094fad8d0cbe776a8403c7"
format_stamp: "Formatted at 2026-08-08 06:04:34 on dist-test-slave-zj8q"
server_key: "4130e9839883e795c1424cf10a682f58"
server_key_iv: "e7d7a1ab61f4043c46d055a61c537c28"
server_key_version: "encryptionkey@0"
I20260808 06:04:34.689065 2492 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 06:04:34.691473 2514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:34.692647 2492 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 06:04:34.692776 2492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "848ae95361094fad8d0cbe776a8403c7"
format_stamp: "Formatted at 2026-08-08 06:04:34 on dist-test-slave-zj8q"
server_key: "4130e9839883e795c1424cf10a682f58"
server_key_iv: "e7d7a1ab61f4043c46d055a61c537c28"
server_key_version: "encryptionkey@0"
I20260808 06:04:34.692870 2492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:34.703158 2492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:34.705693 2492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:34.705850 2492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:34.712267 2492 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:45283
I20260808 06:04:34.712316 2591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:45283 every 8 connection(s)
I20260808 06:04:34.713124 2492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:04:34.715780 2592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:34.720819 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 2492
I20260808 06:04:34.720930 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:04:34.721176 1630 external_mini_cluster.cc:1428] Setting key 6b1ac3a9b2a9cdbfeb6866db20420572
I20260808 06:04:34.721233 2592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 848ae95361094fad8d0cbe776a8403c7: Bootstrap starting.
I20260808 06:04:34.723943 2592 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 848ae95361094fad8d0cbe776a8403c7: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:34.724822 2592 log.cc:824] T 00000000000000000000000000000000 P 848ae95361094fad8d0cbe776a8403c7: Log is configured to *not* fsync() on all Append() calls
I20260808 06:04:34.726891 2592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 848ae95361094fad8d0cbe776a8403c7: No bootstrap required, opened a new log
Aug 08 06:04:34 dist-test-slave-zj8q krb5kdc[2459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:04:34.730854 2592 raft_consensus.cc:348] T 00000000000000000000000000000000 P 848ae95361094fad8d0cbe776a8403c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "848ae95361094fad8d0cbe776a8403c7" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 45283 } }
I20260808 06:04:34.731062 2592 raft_consensus.cc:374] T 00000000000000000000000000000000 P 848ae95361094fad8d0cbe776a8403c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:34.731123 2592 raft_consensus.cc:729] T 00000000000000000000000000000000 P 848ae95361094fad8d0cbe776a8403c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 848ae95361094fad8d0cbe776a8403c7, State: Initialized, Role: FOLLOWER
I20260808 06:04:34.731537 2592 consensus_queue.cc:260] T 00000000000000000000000000000000 P 848ae95361094fad8d0cbe776a8403c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "848ae95361094fad8d0cbe776a8403c7" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 45283 } }
I20260808 06:04:34.731683 2592 raft_consensus.cc:388] T 00000000000000000000000000000000 P 848ae95361094fad8d0cbe776a8403c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:04:34.731755 2592 raft_consensus.cc:482] T 00000000000000000000000000000000 P 848ae95361094fad8d0cbe776a8403c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:04:34.731854 2592 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 848ae95361094fad8d0cbe776a8403c7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:34.732649 2592 raft_consensus.cc:504] T 00000000000000000000000000000000 P 848ae95361094fad8d0cbe776a8403c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "848ae95361094fad8d0cbe776a8403c7" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 45283 } }
I20260808 06:04:34.733012 2592 leader_election.cc:302] T 00000000000000000000000000000000 P 848ae95361094fad8d0cbe776a8403c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 848ae95361094fad8d0cbe776a8403c7; no voters:
I20260808 06:04:34.733318 2592 leader_election.cc:288] T 00000000000000000000000000000000 P 848ae95361094fad8d0cbe776a8403c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:04:34.733376 2602 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 848ae95361094fad8d0cbe776a8403c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:04:34.733666 2602 raft_consensus.cc:686] T 00000000000000000000000000000000 P 848ae95361094fad8d0cbe776a8403c7 [term 1 LEADER]: Becoming Leader. State: Replica: 848ae95361094fad8d0cbe776a8403c7, State: Running, Role: LEADER
I20260808 06:04:34.733958 2602 consensus_queue.cc:237] T 00000000000000000000000000000000 P 848ae95361094fad8d0cbe776a8403c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "848ae95361094fad8d0cbe776a8403c7" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 45283 } }
I20260808 06:04:34.734589 2592 sys_catalog.cc:564] T 00000000000000000000000000000000 P 848ae95361094fad8d0cbe776a8403c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:04:34.735567 2603 sys_catalog.cc:455] T 00000000000000000000000000000000 P 848ae95361094fad8d0cbe776a8403c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "848ae95361094fad8d0cbe776a8403c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "848ae95361094fad8d0cbe776a8403c7" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 45283 } } }
I20260808 06:04:34.735714 2603 sys_catalog.cc:458] T 00000000000000000000000000000000 P 848ae95361094fad8d0cbe776a8403c7 [sys.catalog]: This master's current role is: LEADER
I20260808 06:04:34.735801 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:34.722455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34010 (local address 127.1.151.190:45283)
0808 06:04:34.722874 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:34.722892 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:34.722933 (+ 41us) server_negotiation.cc:407] Connection header received
0808 06:04:34.723710 (+ 777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:34.723733 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:34.724162 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:34.724517 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:34.725184 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:34.726546 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:34.727412 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:34.728069 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:34.728247 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:34.730826 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:34.730831 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:34.730843 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:34.730868 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:34.732540 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:34.733006 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:34.733010 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:34.733015 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:34.733162 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:34.733481 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:34.733484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:34.733486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:34.733968 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:34.734193 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:34.734541 (+ 348us) server_negotiation.cc:299] Negotiation successful
0808 06:04:34.734834 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":105,"threads_started":1}
I20260808 06:04:34.735944 2603 sys_catalog.cc:455] T 00000000000000000000000000000000 P 848ae95361094fad8d0cbe776a8403c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 848ae95361094fad8d0cbe776a8403c7. Latest consensus state: current_term: 1 leader_uuid: "848ae95361094fad8d0cbe776a8403c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "848ae95361094fad8d0cbe776a8403c7" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 45283 } } }
I20260808 06:04:34.736019 2603 sys_catalog.cc:458] T 00000000000000000000000000000000 P 848ae95361094fad8d0cbe776a8403c7 [sys.catalog]: This master's current role is: LEADER
I20260808 06:04:34.737788 2617 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:04:34.740324 2617 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:04:34.745709 2617 catalog_manager.cc:1347] Generated new cluster ID: 3ae7a16b2100462dbd66b1b2ee658cd2
I20260808 06:04:34.745772 2617 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:04:34.763906 2617 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:04:34.765117 2617 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:04:34.781886 2617 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 848ae95361094fad8d0cbe776a8403c7: Generated new TSK 0
I20260808 06:04:34.782606 2617 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.151.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab.
Entry for principal kudu/127.1.151.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab.
I20260808 06:04:34.810667 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.129:0
--local_ip_for_outbound_sockets=127.1.151.129
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:45283
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:43235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:04:34.920715 2635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:34.920946 2635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:34.920989 2635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:34.924899 2635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:34.924952 2635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:34.925028 2635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.129
I20260808 06:04:34.930119 2635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:43235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:45283
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.129
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:34.931142 2635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:34.932168 2635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:34.941670 2649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:34.941641 2652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:34.941646 2650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:34.941828 2635 server_base.cc:1034] running on GCE node
I20260808 06:04:34.942365 2635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:34.942988 2635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:34.944164 2635 hybrid_clock.cc:648] HybridClock initialized: now 1786169074944147 us; error 30 us; skew 500 ppm
Aug 08 06:04:34 dist-test-slave-zj8q krb5kdc[2459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169074, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:04:34.946831 2635 init.cc:377] Logged in from keytab as kudu/127.1.151.129@KRBTEST.COM (short username kudu)
I20260808 06:04:34.947675 2635 webserver.cc:460] Webserver started at http://127.1.151.129:36445/ using document root <none> and password file <none>
I20260808 06:04:34.948213 2635 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:34.948280 2635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:34.948477 2635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:34.950519 2635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-0/data/instance:
uuid: "3ec1c8c659c74ac0b756af8fbdb301f4"
format_stamp: "Formatted at 2026-08-08 06:04:34 on dist-test-slave-zj8q"
server_key: "a4edb31c0b9988890c699eb0df5e17cf"
server_key_iv: "4972ea0406a92e959fbd3050da4a318c"
server_key_version: "encryptionkey@0"
I20260808 06:04:34.951066 2635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance:
uuid: "3ec1c8c659c74ac0b756af8fbdb301f4"
format_stamp: "Formatted at 2026-08-08 06:04:34 on dist-test-slave-zj8q"
server_key: "a4edb31c0b9988890c699eb0df5e17cf"
server_key_iv: "4972ea0406a92e959fbd3050da4a318c"
server_key_version: "encryptionkey@0"
I20260808 06:04:34.954672 2635 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 06:04:34.956951 2664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:34.958117 2635 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:04:34.958235 2635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-0/wal
uuid: "3ec1c8c659c74ac0b756af8fbdb301f4"
format_stamp: "Formatted at 2026-08-08 06:04:34 on dist-test-slave-zj8q"
server_key: "a4edb31c0b9988890c699eb0df5e17cf"
server_key_iv: "4972ea0406a92e959fbd3050da4a318c"
server_key_version: "encryptionkey@0"
I20260808 06:04:34.958320 2635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:34.971503 2635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:34.974119 2635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:34.974310 2635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:34.974912 2635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:04:34.975680 2635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:04:34.975723 2635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:34.975760 2635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:04:34.975780 2635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:34.983660 2635 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.129:44021
I20260808 06:04:34.983729 2835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.129:44021 every 8 connection(s)
I20260808 06:04:34.984498 2635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
I20260808 06:04:34.986516 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 2635
I20260808 06:04:34.986629 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance
I20260808 06:04:34.986869 1630 external_mini_cluster.cc:1428] Setting key 8ec7993621b3a2a32643b49af5743de5
Aug 08 06:04:34 dist-test-slave-zj8q krb5kdc[2459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169074, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.151.130@KRBTEST.COM; defaulting to no policy
I20260808 06:04:34.998090 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:34.986381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:40299 (local address 127.1.151.190:45283)
0808 06:04:34.986536 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:34.986541 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:34.987432 (+ 891us) server_negotiation.cc:407] Connection header received
0808 06:04:34.988951 (+ 1519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:34.988954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:34.989005 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:34.989076 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:34.990275 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:34.990809 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:34.991913 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:34.992152 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:34.992239 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:34.994565 (+ 2326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:34.994570 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:34.994582 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:34.994611 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:34.996142 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:34.996642 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:34.996645 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:34.996647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:34.996758 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:34.997080 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:34.997083 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:34.997084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:34.997260 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:34.997340 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:34.997840 (+ 500us) server_negotiation.cc:299] Negotiation successful
0808 06:04:34.997951 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 06:04:34.998847 2838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:34.986793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:45283 (local address 127.1.151.129:40299)
0808 06:04:34.987296 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:34.987351 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:34.988753 (+ 1402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:34.989272 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:34.989280 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:34.989640 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:34.990125 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:34.990140 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:34.990925 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:34.990929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:34.991797 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:34.991807 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:34.992285 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:34.992289 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:34.992454 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:34.992972 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:04:34.992987 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:34.994399 (+ 1412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:34.996268 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:34.996275 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:34.996289 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:34.996530 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:34.996848 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:34.996851 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:34.996853 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:34.996983 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:34.997350 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:34.997354 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:34.997584 (+ 230us) client_negotiation.cc:769] Sending connection context
0808 06:04:34.997745 (+ 161us) client_negotiation.cc:240] Negotiation successful
0808 06:04:34.997981 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":126,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.130@KRBTEST.COM" created.
I20260808 06:04:35.000064 2836 heartbeater.cc:347] Connected to a master server at 127.1.151.190:45283
I20260808 06:04:35.000311 2836 heartbeater.cc:464] Registering TS with master...
I20260808 06:04:35.000866 2836 heartbeater.cc:511] Master 127.1.151.190:45283 requested a full tablet report, sending...
I20260808 06:04:35.002394 2537 ts_manager.cc:194] Registered new tserver with Master: 3ec1c8c659c74ac0b756af8fbdb301f4 (127.1.151.129:44021)
I20260808 06:04:35.003505 2537 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.129@KRBTEST.COM'} at 127.1.151.129:40299
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab.
Entry for principal kudu/127.1.151.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab.
I20260808 06:04:35.012723 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.130:0
--local_ip_for_outbound_sockets=127.1.151.130
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:45283
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:43235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:04:35.131960 2845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:35.132211 2845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:35.132269 2845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:35.136286 2845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:35.136338 2845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:35.136415 2845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.130
I20260808 06:04:35.141486 2845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:43235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:45283
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.130
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:35.142490 2845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:35.143445 2845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:35.152707 2855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:35.152712 2856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:35.152806 2845 server_base.cc:1034] running on GCE node
W20260808 06:04:35.152942 2859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:35.153297 2845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:35.153865 2845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:35.155019 2845 hybrid_clock.cc:648] HybridClock initialized: now 1786169075154993 us; error 38 us; skew 500 ppm
Aug 08 06:04:35 dist-test-slave-zj8q krb5kdc[2459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169075, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:04:35.157604 2845 init.cc:377] Logged in from keytab as kudu/127.1.151.130@KRBTEST.COM (short username kudu)
I20260808 06:04:35.158453 2845 webserver.cc:460] Webserver started at http://127.1.151.130:45097/ using document root <none> and password file <none>
I20260808 06:04:35.159029 2845 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:35.159094 2845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:35.159315 2845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:35.161255 2845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-1/data/instance:
uuid: "8577dd8a924f474a987dec2e376c5701"
format_stamp: "Formatted at 2026-08-08 06:04:35 on dist-test-slave-zj8q"
server_key: "eed57ff5cc3f3de0e52066bf8bee4056"
server_key_iv: "ef58e89e85859ead4ea99fdc17fd6b48"
server_key_version: "encryptionkey@0"
I20260808 06:04:35.161821 2845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance:
uuid: "8577dd8a924f474a987dec2e376c5701"
format_stamp: "Formatted at 2026-08-08 06:04:35 on dist-test-slave-zj8q"
server_key: "eed57ff5cc3f3de0e52066bf8bee4056"
server_key_iv: "ef58e89e85859ead4ea99fdc17fd6b48"
server_key_version: "encryptionkey@0"
I20260808 06:04:35.166291 2845 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260808 06:04:35.168797 2870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:35.170011 2845 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:04:35.170121 2845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-1/wal
uuid: "8577dd8a924f474a987dec2e376c5701"
format_stamp: "Formatted at 2026-08-08 06:04:35 on dist-test-slave-zj8q"
server_key: "eed57ff5cc3f3de0e52066bf8bee4056"
server_key_iv: "ef58e89e85859ead4ea99fdc17fd6b48"
server_key_version: "encryptionkey@0"
I20260808 06:04:35.170276 2845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:35.187110 2845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:35.190043 2845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:35.190234 2845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:35.190789 2845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:04:35.191591 2845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:04:35.191639 2845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:35.191679 2845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:04:35.191694 2845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:35.200423 2845 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.130:46789
I20260808 06:04:35.200488 3058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.130:46789 every 8 connection(s)
I20260808 06:04:35.201300 2845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
I20260808 06:04:35.209623 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 2845
I20260808 06:04:35.209725 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance
I20260808 06:04:35.209978 1630 external_mini_cluster.cc:1428] Setting key c4ff55dfe61517cacf0a4c95a1c46a7c
Aug 08 06:04:35 dist-test-slave-zj8q krb5kdc[2459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169075, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:04:35.214625 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:35.203125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:59663 (local address 127.1.151.190:45283)
0808 06:04:35.203254 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:35.203258 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:35.204121 (+ 863us) server_negotiation.cc:407] Connection header received
0808 06:04:35.205450 (+ 1329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:35.205453 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:35.205508 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:35.205594 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:35.206890 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:35.207421 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:35.208563 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:35.208785 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:35.208873 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:35.211335 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:35.211339 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:35.211342 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:35.211368 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:35.212644 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:35.213123 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:35.213127 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:35.213129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:35.213222 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:35.213541 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:35.213544 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:35.213546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:35.213727 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:35.213815 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:35.214336 (+ 521us) server_negotiation.cc:299] Negotiation successful
0808 06:04:35.214468 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 06:04:35.215534 3061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:35.203516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:45283 (local address 127.1.151.130:59663)
0808 06:04:35.203978 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:35.204019 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:35.205245 (+ 1226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:35.205808 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:35.205814 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:35.206205 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:35.206747 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:35.206762 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:35.207539 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:35.207542 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:35.208442 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:35.208451 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:35.208924 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:35.208928 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:35.209094 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:35.209614 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:04:35.209631 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:35.211182 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:35.212759 (+ 1577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:35.212765 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:35.212780 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:35.213027 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:35.213307 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:35.213310 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:35.213312 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:35.213452 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:35.213828 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:35.213835 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:35.214067 (+ 232us) client_negotiation.cc:769] Sending connection context
0808 06:04:35.214263 (+ 196us) client_negotiation.cc:240] Negotiation successful
0808 06:04:35.214571 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":116,"threads_started":1}
I20260808 06:04:35.216554 3059 heartbeater.cc:347] Connected to a master server at 127.1.151.190:45283
I20260808 06:04:35.216787 3059 heartbeater.cc:464] Registering TS with master...
I20260808 06:04:35.217236 3059 heartbeater.cc:511] Master 127.1.151.190:45283 requested a full tablet report, sending...
I20260808 06:04:35.218314 2537 ts_manager.cc:194] Registered new tserver with Master: 8577dd8a924f474a987dec2e376c5701 (127.1.151.130:46789)
I20260808 06:04:35.219046 2537 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.130@KRBTEST.COM'} at 127.1.151.130:59663
WARNING: no policy specified for kudu/127.1.151.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab.
Entry for principal kudu/127.1.151.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab.
I20260808 06:04:35.237422 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.131:0
--local_ip_for_outbound_sockets=127.1.151.131
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:45283
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:43235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:04:35.346110 3065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:35.346359 3065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:35.346400 3065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:35.350498 3065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:35.350549 3065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:35.350626 3065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.131
I20260808 06:04:35.355644 3065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:43235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:45283
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.131
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:35.356565 3065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:35.357487 3065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:35.366647 3071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:35.366627 3074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:35.366667 3072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:35.366780 3065 server_base.cc:1034] running on GCE node
I20260808 06:04:35.367285 3065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:35.367871 3065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:35.369040 3065 hybrid_clock.cc:648] HybridClock initialized: now 1786169075369017 us; error 33 us; skew 500 ppm
Aug 08 06:04:35 dist-test-slave-zj8q krb5kdc[2459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169075, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:04:35.371717 3065 init.cc:377] Logged in from keytab as kudu/127.1.151.131@KRBTEST.COM (short username kudu)
I20260808 06:04:35.372596 3065 webserver.cc:460] Webserver started at http://127.1.151.131:34531/ using document root <none> and password file <none>
I20260808 06:04:35.373113 3065 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:35.373172 3065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:35.373358 3065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:35.375366 3065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-2/data/instance:
uuid: "dceffcb270b04d60abd93479296a3976"
format_stamp: "Formatted at 2026-08-08 06:04:35 on dist-test-slave-zj8q"
server_key: "17da9b296be1c0c6d8bd32dffacc3e97"
server_key_iv: "b727b76927a8fa7d7e8f193a4a2099c1"
server_key_version: "encryptionkey@0"
I20260808 06:04:35.375897 3065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance:
uuid: "dceffcb270b04d60abd93479296a3976"
format_stamp: "Formatted at 2026-08-08 06:04:35 on dist-test-slave-zj8q"
server_key: "17da9b296be1c0c6d8bd32dffacc3e97"
server_key_iv: "b727b76927a8fa7d7e8f193a4a2099c1"
server_key_version: "encryptionkey@0"
I20260808 06:04:35.379340 3065 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 06:04:35.381604 3086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:35.382727 3065 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 06:04:35.382827 3065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-2/wal
uuid: "dceffcb270b04d60abd93479296a3976"
format_stamp: "Formatted at 2026-08-08 06:04:35 on dist-test-slave-zj8q"
server_key: "17da9b296be1c0c6d8bd32dffacc3e97"
server_key_iv: "b727b76927a8fa7d7e8f193a4a2099c1"
server_key_version: "encryptionkey@0"
I20260808 06:04:35.382921 3065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:35.396379 3065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:35.399140 3065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:35.399302 3065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:35.399876 3065 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:04:35.400666 3065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:04:35.400712 3065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:35.400753 3065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:04:35.400781 3065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:35.409029 3065 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.131:34013
I20260808 06:04:35.409057 3279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.131:34013 every 8 connection(s)
I20260808 06:04:35.409888 3065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
I20260808 06:04:35.412791 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 3065
I20260808 06:04:35.412928 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance
I20260808 06:04:35.413228 1630 external_mini_cluster.cc:1428] Setting key 3df0b10341cbeaecf29718f5d0e614bd
Aug 08 06:04:35 dist-test-slave-zj8q krb5kdc[2459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169075, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:04:35.422606 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:35.411641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:53921 (local address 127.1.151.190:45283)
0808 06:04:35.411777 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:35.411781 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:35.412528 (+ 747us) server_negotiation.cc:407] Connection header received
0808 06:04:35.414065 (+ 1537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:35.414069 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:35.414131 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:35.414229 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:35.415592 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:35.416128 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:35.417200 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:35.417410 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:35.417484 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:35.419700 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:35.419704 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:35.419706 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:35.419728 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:35.420976 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:35.421415 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:35.421417 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:35.421418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:35.421484 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:35.421733 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:35.421735 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:35.421736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:35.421894 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:35.421963 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:35.422401 (+ 438us) server_negotiation.cc:299] Negotiation successful
0808 06:04:35.422490 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 06:04:35.423344 3288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:35.411951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:45283 (local address 127.1.151.131:53921)
0808 06:04:35.412383 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:35.412423 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:35.413836 (+ 1413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:35.414375 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:35.414384 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:35.414824 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:35.415459 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:35.415476 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:35.416228 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:35.416231 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:35.417094 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:35.417113 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:35.417515 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:35.417517 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:35.417651 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:35.418182 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:04:35.418202 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:35.419557 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:35.421079 (+ 1522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:35.421090 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:35.421100 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:35.421340 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:35.421545 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:35.421547 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:35.421548 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:35.421667 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:35.421963 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:35.421967 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:35.422211 (+ 244us) client_negotiation.cc:769] Sending connection context
0808 06:04:35.422366 (+ 155us) client_negotiation.cc:240] Negotiation successful
0808 06:04:35.422597 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":116,"threads_started":1}
I20260808 06:04:35.424327 3280 heartbeater.cc:347] Connected to a master server at 127.1.151.190:45283
I20260808 06:04:35.424523 3280 heartbeater.cc:464] Registering TS with master...
I20260808 06:04:35.424957 3280 heartbeater.cc:511] Master 127.1.151.190:45283 requested a full tablet report, sending...
I20260808 06:04:35.425933 2537 ts_manager.cc:194] Registered new tserver with Master: dceffcb270b04d60abd93479296a3976 (127.1.151.131:34013)
I20260808 06:04:35.426635 2537 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.131@KRBTEST.COM'} at 127.1.151.131:53921
I20260808 06:04:35.427000 1630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 06:04:35.436645 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:35.430192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34024 (local address 127.1.151.190:45283)
0808 06:04:35.430279 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:35.430282 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:35.430440 (+ 158us) server_negotiation.cc:407] Connection header received
0808 06:04:35.430535 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:35.430538 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:35.430570 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:35.430656 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:35.431261 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:35.431742 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:35.432734 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:35.433043 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:35.433124 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:35.434246 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:35.434248 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:35.434250 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:35.434271 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:35.435330 (+ 1059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:35.435723 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:35.435727 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:35.435729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:35.435806 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:35.436009 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:35.436011 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:35.436012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:35.436190 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:35.436286 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:35.436434 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 06:04:35.436515 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 06:04:35.440663 2537 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34024:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 06:04:35.443547 2537 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 06:04:35.457180 3310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:35.452386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33140 (local address 127.1.151.131:34013)
0808 06:04:35.452754 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:35.452764 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:35.452783 (+ 19us) server_negotiation.cc:407] Connection header received
0808 06:04:35.452842 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:35.452847 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:35.452985 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:35.453277 (+ 292us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:04:35.453429 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:35.454191 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:35.455589 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:35.456394 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:35.456676 (+ 282us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:35.456818 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:35.456936 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 06:04:35.457023 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":95,"threads_started":1}
I20260808 06:04:35.457176 3303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:35.451640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:34013 (local address 127.0.0.1:33140)
0808 06:04:35.452272 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:35.452284 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:35.452354 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:35.453145 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:35.453148 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:35.453166 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:04:35.453346 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:35.453354 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:35.454320 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:35.454322 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:35.455443 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:35.455453 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:35.456551 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:35.456555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:35.456743 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:35.456795 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 06:04:35.456908 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 06:04:35.457038 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"mutex_wait_us":99,"thread_start_us":91,"threads_started":1}
I20260808 06:04:35.457442 3309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:35.452035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43328 (local address 127.1.151.130:46789)
0808 06:04:35.452598 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:35.452621 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:35.452646 (+ 25us) server_negotiation.cc:407] Connection header received
0808 06:04:35.452722 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:35.452727 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:35.452916 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:35.453076 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:04:35.453458 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:35.454696 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:35.456019 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:35.456622 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:35.457015 (+ 393us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:35.457162 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:35.457215 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 06:04:35.457303 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":104,"threads_started":1}
I20260808 06:04:35.457476 3308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:35.451410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:46789 (local address 127.0.0.1:43328)
0808 06:04:35.451976 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:35.451989 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:35.452094 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:35.453076 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:35.453082 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:35.453119 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:04:35.453333 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:35.453339 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:35.454842 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:35.454846 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:35.455906 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:35.455916 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:35.456757 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:35.456761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:35.456839 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:35.456859 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 06:04:35.456901 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 06:04:35.457363 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":424,"spinlock_wait_cycles":5888,"thread_start_us":98,"threads_started":1}
I20260808 06:04:35.459271 3307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:35.451275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:44021 (local address 127.0.0.1:45900)
0808 06:04:35.451915 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:35.451958 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:35.452075 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:35.453076 (+ 1001us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:35.453082 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:35.453117 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:04:35.453300 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:35.453307 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:35.456032 (+ 2725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:35.456035 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:35.457074 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:35.457084 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:35.458884 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:35.458888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:35.458978 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:35.459001 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 06:04:35.459074 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 06:04:35.459146 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"mutex_wait_us":4,"spinlock_wait_cycles":6912,"thread_start_us":62,"threads_started":1}
I20260808 06:04:35.459527 3305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:35.451411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45900 (local address 127.1.151.129:44021)
0808 06:04:35.452479 (+ 1068us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:35.452492 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:35.452518 (+ 26us) server_negotiation.cc:407] Connection header received
0808 06:04:35.452594 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:35.452600 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:35.452934 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:35.453450 (+ 516us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:04:35.453601 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:35.454714 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:35.457222 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:35.458752 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:35.458861 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:35.458985 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:35.459140 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 06:04:35.459228 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":958,"thread_start_us":104,"threads_started":1}
I20260808 06:04:35.460182 2945 tablet_service.cc:1467] Processing CreateTablet for tablet 57a8bd79a0db47e2bd3029a85862b359 (DEFAULT_TABLE table=test-table [id=84ab207613f24ff1b3c34b5b70286f8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:04:35.460237 3174 tablet_service.cc:1467] Processing CreateTablet for tablet 57a8bd79a0db47e2bd3029a85862b359 (DEFAULT_TABLE table=test-table [id=84ab207613f24ff1b3c34b5b70286f8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:04:35.461262 3174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57a8bd79a0db47e2bd3029a85862b359. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:35.461283 2945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57a8bd79a0db47e2bd3029a85862b359. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:35.462031 2743 tablet_service.cc:1467] Processing CreateTablet for tablet 57a8bd79a0db47e2bd3029a85862b359 (DEFAULT_TABLE table=test-table [id=84ab207613f24ff1b3c34b5b70286f8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:04:35.463007 2743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57a8bd79a0db47e2bd3029a85862b359. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:35.467182 3311 tablet_bootstrap.cc:492] T 57a8bd79a0db47e2bd3029a85862b359 P dceffcb270b04d60abd93479296a3976: Bootstrap starting.
I20260808 06:04:35.468837 3312 tablet_bootstrap.cc:492] T 57a8bd79a0db47e2bd3029a85862b359 P 8577dd8a924f474a987dec2e376c5701: Bootstrap starting.
I20260808 06:04:35.469589 3316 tablet_bootstrap.cc:492] T 57a8bd79a0db47e2bd3029a85862b359 P 3ec1c8c659c74ac0b756af8fbdb301f4: Bootstrap starting.
I20260808 06:04:35.470419 3311 tablet_bootstrap.cc:654] T 57a8bd79a0db47e2bd3029a85862b359 P dceffcb270b04d60abd93479296a3976: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:35.471185 3312 tablet_bootstrap.cc:654] T 57a8bd79a0db47e2bd3029a85862b359 P 8577dd8a924f474a987dec2e376c5701: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:35.471290 3311 log.cc:824] T 57a8bd79a0db47e2bd3029a85862b359 P dceffcb270b04d60abd93479296a3976: Log is configured to *not* fsync() on all Append() calls
I20260808 06:04:35.471719 3316 tablet_bootstrap.cc:654] T 57a8bd79a0db47e2bd3029a85862b359 P 3ec1c8c659c74ac0b756af8fbdb301f4: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:35.471940 3312 log.cc:824] T 57a8bd79a0db47e2bd3029a85862b359 P 8577dd8a924f474a987dec2e376c5701: Log is configured to *not* fsync() on all Append() calls
I20260808 06:04:35.473088 3316 log.cc:824] T 57a8bd79a0db47e2bd3029a85862b359 P 3ec1c8c659c74ac0b756af8fbdb301f4: Log is configured to *not* fsync() on all Append() calls
I20260808 06:04:35.473515 3311 tablet_bootstrap.cc:492] T 57a8bd79a0db47e2bd3029a85862b359 P dceffcb270b04d60abd93479296a3976: No bootstrap required, opened a new log
I20260808 06:04:35.473701 3311 ts_tablet_manager.cc:1397] T 57a8bd79a0db47e2bd3029a85862b359 P dceffcb270b04d60abd93479296a3976: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.004s
I20260808 06:04:35.473837 3312 tablet_bootstrap.cc:492] T 57a8bd79a0db47e2bd3029a85862b359 P 8577dd8a924f474a987dec2e376c5701: No bootstrap required, opened a new log
I20260808 06:04:35.474020 3312 ts_tablet_manager.cc:1397] T 57a8bd79a0db47e2bd3029a85862b359 P 8577dd8a924f474a987dec2e376c5701: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 06:04:35.475292 3316 tablet_bootstrap.cc:492] T 57a8bd79a0db47e2bd3029a85862b359 P 3ec1c8c659c74ac0b756af8fbdb301f4: No bootstrap required, opened a new log
I20260808 06:04:35.475466 3316 ts_tablet_manager.cc:1397] T 57a8bd79a0db47e2bd3029a85862b359 P 3ec1c8c659c74ac0b756af8fbdb301f4: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 06:04:35.476949 3312 raft_consensus.cc:348] T 57a8bd79a0db47e2bd3029a85862b359 P 8577dd8a924f474a987dec2e376c5701 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ec1c8c659c74ac0b756af8fbdb301f4" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 44021 } } peers { permanent_uuid: "8577dd8a924f474a987dec2e376c5701" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 46789 } } peers { permanent_uuid: "dceffcb270b04d60abd93479296a3976" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34013 } }
I20260808 06:04:35.477170 3312 raft_consensus.cc:374] T 57a8bd79a0db47e2bd3029a85862b359 P 8577dd8a924f474a987dec2e376c5701 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:35.477231 3312 raft_consensus.cc:729] T 57a8bd79a0db47e2bd3029a85862b359 P 8577dd8a924f474a987dec2e376c5701 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8577dd8a924f474a987dec2e376c5701, State: Initialized, Role: FOLLOWER
I20260808 06:04:35.477627 3312 consensus_queue.cc:260] T 57a8bd79a0db47e2bd3029a85862b359 P 8577dd8a924f474a987dec2e376c5701 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ec1c8c659c74ac0b756af8fbdb301f4" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 44021 } } peers { permanent_uuid: "8577dd8a924f474a987dec2e376c5701" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 46789 } } peers { permanent_uuid: "dceffcb270b04d60abd93479296a3976" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34013 } }
I20260808 06:04:35.477643 3311 raft_consensus.cc:348] T 57a8bd79a0db47e2bd3029a85862b359 P dceffcb270b04d60abd93479296a3976 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ec1c8c659c74ac0b756af8fbdb301f4" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 44021 } } peers { permanent_uuid: "8577dd8a924f474a987dec2e376c5701" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 46789 } } peers { permanent_uuid: "dceffcb270b04d60abd93479296a3976" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34013 } }
I20260808 06:04:35.477893 3311 raft_consensus.cc:374] T 57a8bd79a0db47e2bd3029a85862b359 P dceffcb270b04d60abd93479296a3976 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:35.477947 3311 raft_consensus.cc:729] T 57a8bd79a0db47e2bd3029a85862b359 P dceffcb270b04d60abd93479296a3976 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dceffcb270b04d60abd93479296a3976, State: Initialized, Role: FOLLOWER
I20260808 06:04:35.478375 3311 consensus_queue.cc:260] T 57a8bd79a0db47e2bd3029a85862b359 P dceffcb270b04d60abd93479296a3976 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ec1c8c659c74ac0b756af8fbdb301f4" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 44021 } } peers { permanent_uuid: "8577dd8a924f474a987dec2e376c5701" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 46789 } } peers { permanent_uuid: "dceffcb270b04d60abd93479296a3976" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34013 } }
I20260808 06:04:35.478533 3312 ts_tablet_manager.cc:1428] T 57a8bd79a0db47e2bd3029a85862b359 P 8577dd8a924f474a987dec2e376c5701: Time spent starting tablet: real 0.004s user 0.001s sys 0.004s
I20260808 06:04:35.478889 3059 heartbeater.cc:503] Master 127.1.151.190:45283 was elected leader, sending a full tablet report...
I20260808 06:04:35.478988 3280 heartbeater.cc:503] Master 127.1.151.190:45283 was elected leader, sending a full tablet report...
I20260808 06:04:35.479015 3316 raft_consensus.cc:348] T 57a8bd79a0db47e2bd3029a85862b359 P 3ec1c8c659c74ac0b756af8fbdb301f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ec1c8c659c74ac0b756af8fbdb301f4" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 44021 } } peers { permanent_uuid: "8577dd8a924f474a987dec2e376c5701" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 46789 } } peers { permanent_uuid: "dceffcb270b04d60abd93479296a3976" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34013 } }
I20260808 06:04:35.479235 3316 raft_consensus.cc:374] T 57a8bd79a0db47e2bd3029a85862b359 P 3ec1c8c659c74ac0b756af8fbdb301f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:35.479283 3316 raft_consensus.cc:729] T 57a8bd79a0db47e2bd3029a85862b359 P 3ec1c8c659c74ac0b756af8fbdb301f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ec1c8c659c74ac0b756af8fbdb301f4, State: Initialized, Role: FOLLOWER
I20260808 06:04:35.479322 3311 ts_tablet_manager.cc:1428] T 57a8bd79a0db47e2bd3029a85862b359 P dceffcb270b04d60abd93479296a3976: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 06:04:35.479717 3316 consensus_queue.cc:260] T 57a8bd79a0db47e2bd3029a85862b359 P 3ec1c8c659c74ac0b756af8fbdb301f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ec1c8c659c74ac0b756af8fbdb301f4" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 44021 } } peers { permanent_uuid: "8577dd8a924f474a987dec2e376c5701" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 46789 } } peers { permanent_uuid: "dceffcb270b04d60abd93479296a3976" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34013 } }
I20260808 06:04:35.480469 2836 heartbeater.cc:503] Master 127.1.151.190:45283 was elected leader, sending a full tablet report...
I20260808 06:04:35.480731 3316 ts_tablet_manager.cc:1428] T 57a8bd79a0db47e2bd3029a85862b359 P 3ec1c8c659c74ac0b756af8fbdb301f4: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260808 06:04:35.485630 2837 tablet.cc:2369] T 57a8bd79a0db47e2bd3029a85862b359 P 3ec1c8c659c74ac0b756af8fbdb301f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 06:04:35.660866 3286 tablet.cc:2369] T 57a8bd79a0db47e2bd3029a85862b359 P dceffcb270b04d60abd93479296a3976: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 06:04:35.702433 3060 tablet.cc:2369] T 57a8bd79a0db47e2bd3029a85862b359 P 8577dd8a924f474a987dec2e376c5701: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 06:04:35.853889 3321 raft_consensus.cc:482] T 57a8bd79a0db47e2bd3029a85862b359 P 8577dd8a924f474a987dec2e376c5701 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:04:35.854115 3321 raft_consensus.cc:504] T 57a8bd79a0db47e2bd3029a85862b359 P 8577dd8a924f474a987dec2e376c5701 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ec1c8c659c74ac0b756af8fbdb301f4" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 44021 } } peers { permanent_uuid: "8577dd8a924f474a987dec2e376c5701" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 46789 } } peers { permanent_uuid: "dceffcb270b04d60abd93479296a3976" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34013 } }
I20260808 06:04:35.855314 3321 leader_election.cc:288] T 57a8bd79a0db47e2bd3029a85862b359 P 8577dd8a924f474a987dec2e376c5701 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ec1c8c659c74ac0b756af8fbdb301f4 (127.1.151.129:44021), dceffcb270b04d60abd93479296a3976 (127.1.151.131:34013)
I20260808 06:04:35.859252 3310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:35.855722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:34469 (local address 127.1.151.131:34013)
0808 06:04:35.855836 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:35.855840 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:35.856187 (+ 347us) server_negotiation.cc:407] Connection header received
0808 06:04:35.856291 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:35.856295 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:35.856348 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:35.856423 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:04:35.856746 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:35.857227 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:35.858280 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:35.858694 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:35.858794 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:35.858848 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:35.859044 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 06:04:35.859120 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 06:04:35.859267 3305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:35.855696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:54403 (local address 127.1.151.129:44021)
0808 06:04:35.855832 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:35.855837 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:35.856058 (+ 221us) server_negotiation.cc:407] Connection header received
0808 06:04:35.856177 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:35.856181 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:35.856233 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:35.856308 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:04:35.856625 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:35.857176 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:35.858315 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:35.858723 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:35.858834 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:35.858885 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:35.859092 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 06:04:35.859167 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 06:04:35.859263 3328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:35.855582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:34013 (local address 127.1.151.130:34469)
0808 06:04:35.856065 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:35.856079 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:35.856201 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:35.856444 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:35.856446 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:35.856460 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:04:35.856636 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:35.856645 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:35.857338 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:35.857340 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:35.858175 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:35.858184 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:35.858815 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:35.858819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:35.858918 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:35.858957 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 06:04:35.859012 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 06:04:35.859077 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"mutex_wait_us":232,"thread_start_us":94,"threads_started":1}
I20260808 06:04:35.859278 3327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:35.855582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:44021 (local address 127.1.151.130:54403)
0808 06:04:35.855948 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:35.855964 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:35.856081 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:35.856328 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:35.856331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:35.856347 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:04:35.856501 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:35.856510 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:35.857288 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:35.857291 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:35.858186 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:35.858198 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:35.858840 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:35.858843 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:35.858972 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:35.858990 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 06:04:35.859066 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 06:04:35.859129 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":39,"thread_start_us":78,"threads_started":1}
I20260808 06:04:35.859931 3212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57a8bd79a0db47e2bd3029a85862b359" candidate_uuid: "8577dd8a924f474a987dec2e376c5701" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dceffcb270b04d60abd93479296a3976" is_pre_election: true
I20260808 06:04:35.859931 2765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57a8bd79a0db47e2bd3029a85862b359" candidate_uuid: "8577dd8a924f474a987dec2e376c5701" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ec1c8c659c74ac0b756af8fbdb301f4" is_pre_election: true
I20260808 06:04:35.860241 2765 raft_consensus.cc:2454] T 57a8bd79a0db47e2bd3029a85862b359 P 3ec1c8c659c74ac0b756af8fbdb301f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8577dd8a924f474a987dec2e376c5701 in term 0.
I20260808 06:04:35.860241 3212 raft_consensus.cc:2454] T 57a8bd79a0db47e2bd3029a85862b359 P dceffcb270b04d60abd93479296a3976 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8577dd8a924f474a987dec2e376c5701 in term 0.
I20260808 06:04:35.860734 2880 leader_election.cc:302] T 57a8bd79a0db47e2bd3029a85862b359 P 8577dd8a924f474a987dec2e376c5701 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8577dd8a924f474a987dec2e376c5701, dceffcb270b04d60abd93479296a3976; no voters:
I20260808 06:04:35.860973 3321 raft_consensus.cc:2781] T 57a8bd79a0db47e2bd3029a85862b359 P 8577dd8a924f474a987dec2e376c5701 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 06:04:35.861055 3321 raft_consensus.cc:482] T 57a8bd79a0db47e2bd3029a85862b359 P 8577dd8a924f474a987dec2e376c5701 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 06:04:35.861106 3321 raft_consensus.cc:3037] T 57a8bd79a0db47e2bd3029a85862b359 P 8577dd8a924f474a987dec2e376c5701 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:35.862228 3321 raft_consensus.cc:504] T 57a8bd79a0db47e2bd3029a85862b359 P 8577dd8a924f474a987dec2e376c5701 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ec1c8c659c74ac0b756af8fbdb301f4" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 44021 } } peers { permanent_uuid: "8577dd8a924f474a987dec2e376c5701" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 46789 } } peers { permanent_uuid: "dceffcb270b04d60abd93479296a3976" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34013 } }
I20260808 06:04:35.862564 3321 leader_election.cc:288] T 57a8bd79a0db47e2bd3029a85862b359 P 8577dd8a924f474a987dec2e376c5701 [CANDIDATE]: Term 1 election: Requested vote from peers 3ec1c8c659c74ac0b756af8fbdb301f4 (127.1.151.129:44021), dceffcb270b04d60abd93479296a3976 (127.1.151.131:34013)
I20260808 06:04:35.862919 3212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57a8bd79a0db47e2bd3029a85862b359" candidate_uuid: "8577dd8a924f474a987dec2e376c5701" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dceffcb270b04d60abd93479296a3976"
I20260808 06:04:35.862950 2765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57a8bd79a0db47e2bd3029a85862b359" candidate_uuid: "8577dd8a924f474a987dec2e376c5701" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ec1c8c659c74ac0b756af8fbdb301f4"
I20260808 06:04:35.863030 3212 raft_consensus.cc:3037] T 57a8bd79a0db47e2bd3029a85862b359 P dceffcb270b04d60abd93479296a3976 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:35.863068 2765 raft_consensus.cc:3037] T 57a8bd79a0db47e2bd3029a85862b359 P 3ec1c8c659c74ac0b756af8fbdb301f4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:35.864019 3212 raft_consensus.cc:2454] T 57a8bd79a0db47e2bd3029a85862b359 P dceffcb270b04d60abd93479296a3976 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8577dd8a924f474a987dec2e376c5701 in term 1.
I20260808 06:04:35.864104 2765 raft_consensus.cc:2454] T 57a8bd79a0db47e2bd3029a85862b359 P 3ec1c8c659c74ac0b756af8fbdb301f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8577dd8a924f474a987dec2e376c5701 in term 1.
I20260808 06:04:35.864320 2880 leader_election.cc:302] T 57a8bd79a0db47e2bd3029a85862b359 P 8577dd8a924f474a987dec2e376c5701 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8577dd8a924f474a987dec2e376c5701, dceffcb270b04d60abd93479296a3976; no voters:
I20260808 06:04:35.864471 3321 raft_consensus.cc:2781] T 57a8bd79a0db47e2bd3029a85862b359 P 8577dd8a924f474a987dec2e376c5701 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:04:35.864694 3321 raft_consensus.cc:686] T 57a8bd79a0db47e2bd3029a85862b359 P 8577dd8a924f474a987dec2e376c5701 [term 1 LEADER]: Becoming Leader. State: Replica: 8577dd8a924f474a987dec2e376c5701, State: Running, Role: LEADER
I20260808 06:04:35.865018 3321 consensus_queue.cc:237] T 57a8bd79a0db47e2bd3029a85862b359 P 8577dd8a924f474a987dec2e376c5701 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ec1c8c659c74ac0b756af8fbdb301f4" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 44021 } } peers { permanent_uuid: "8577dd8a924f474a987dec2e376c5701" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 46789 } } peers { permanent_uuid: "dceffcb270b04d60abd93479296a3976" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34013 } }
I20260808 06:04:35.868213 2537 catalog_manager.cc:5495] T 57a8bd79a0db47e2bd3029a85862b359 P 8577dd8a924f474a987dec2e376c5701 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8577dd8a924f474a987dec2e376c5701 (127.1.151.130). New cstate: current_term: 1 leader_uuid: "8577dd8a924f474a987dec2e376c5701" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ec1c8c659c74ac0b756af8fbdb301f4" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 44021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8577dd8a924f474a987dec2e376c5701" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 46789 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dceffcb270b04d60abd93479296a3976" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34013 } health_report { overall_health: UNKNOWN } } }
Aug 08 06:04:35 dist-test-slave-zj8q krb5kdc[2459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.129@KRBTEST.COM
I20260808 06:04:35.910066 3305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:35.901246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45916 (local address 127.1.151.129:44021)
0808 06:04:35.901396 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:35.901400 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:35.901531 (+ 131us) server_negotiation.cc:407] Connection header received
0808 06:04:35.901738 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:35.901742 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:35.901787 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:35.901853 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:35.902597 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:35.903111 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:35.903959 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:35.904184 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:35.904272 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:35.906797 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:35.906808 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:35.906824 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:35.906849 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:35.908334 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:35.908764 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:35.908770 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:35.908771 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:35.908864 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:35.909120 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:35.909123 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:35.909125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:35.909379 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:35.909588 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:35.909800 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 06:04:35.909913 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 06:04:35 dist-test-slave-zj8q krb5kdc[2459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169075, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 06:04:35 dist-test-slave-zj8q krb5kdc[2459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169075, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.151.129@KRBTEST.COM
I20260808 06:04:35.936949 3305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:35.929021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45920 (local address 127.1.151.129:44021)
0808 06:04:35.929171 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:35.929175 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:35.929270 (+ 95us) server_negotiation.cc:407] Connection header received
0808 06:04:35.929396 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:35.929400 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:35.929441 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:35.929506 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:35.930185 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:35.930684 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:35.931478 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:35.931693 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:35.931779 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:35.933905 (+ 2126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:35.933908 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:35.933910 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:35.933933 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:35.935412 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:35.935911 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:35.935914 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:35.935915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:35.935993 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:35.936204 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:35.936206 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:35.936207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:35.936348 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:35.936441 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:35.936628 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 06:04:35.936743 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 06:04:35 dist-test-slave-zj8q krb5kdc[2459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169075, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 06:04:35 dist-test-slave-zj8q krb5kdc[2459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169075, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.129@KRBTEST.COM
I20260808 06:04:35.961516 3305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:35.954210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45930 (local address 127.1.151.129:44021)
0808 06:04:35.954324 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:35.954327 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:35.954499 (+ 172us) server_negotiation.cc:407] Connection header received
0808 06:04:35.954619 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:35.954621 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:35.954684 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:35.954743 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:35.955331 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:35.955834 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:35.956601 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:35.956818 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:35.956888 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:35.959061 (+ 2173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:35.959064 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:35.959066 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:35.959090 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:35.960241 (+ 1151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:35.960580 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:35.960583 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:35.960584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:35.960668 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:35.960860 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:35.960862 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:35.960864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:35.961006 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:35.961092 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:35.961287 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 06:04:35.961392 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260808 06:04:35.963055 2709 tablet_service.cc:2908] Rejecting scan request for tablet 57a8bd79a0db47e2bd3029a85862b359: Uninitialized: clean time has not yet been initialized
I20260808 06:04:35.964964 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 2635
I20260808 06:04:35.976733 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 2845
I20260808 06:04:35.989815 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 3065
I20260808 06:04:35.999789 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 2492
2026-08-08T06:04:36Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2726 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:04:36 dist-test-slave-zj8q krb5kdc[3351](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:04:36 dist-test-slave-zj8q krb5kdc[3351](info): set up 2 sockets
Aug 08 06:04:36 dist-test-slave-zj8q krb5kdc[3351](info): commencing operation
krb5kdc: starting...
W20260808 06:04:37.148365 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:04:37 dist-test-slave-zj8q krb5kdc[3351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:04:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:04:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.151.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
I20260808 06:04:37.266925 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:39871
--webserver_interface=127.1.151.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:35451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:39871
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:04:37.374925 3375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:37.375155 3375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:37.375193 3375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:37.379040 3375 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:04:37.379088 3375 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 06:04:37.379103 3375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:37.379123 3375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:37.379154 3375 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:04:37.379179 3375 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:04:37.384168 3375 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:35451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:39871
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:39871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:37.385084 3375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:37.386065 3375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:37.394393 3382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:37.394425 3381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:37.394449 3375 server_base.cc:1034] running on GCE node
W20260808 06:04:37.394402 3384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:37.395184 3375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:37.396179 3375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:37.397336 3375 hybrid_clock.cc:648] HybridClock initialized: now 1786169077397309 us; error 43 us; skew 500 ppm
Aug 08 06:04:37 dist-test-slave-zj8q krb5kdc[3351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169077, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:04:37.400295 3375 init.cc:377] Logged in from keytab as kudu/127.1.151.190@KRBTEST.COM (short username kudu)
I20260808 06:04:37.401201 3375 webserver.cc:460] Webserver started at http://127.1.151.190:44453/ using document root <none> and password file <none>
I20260808 06:04:37.401739 3375 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:37.401814 3375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:37.401991 3375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:37.403986 3375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "00147b47a31b4ab5a2080efc0f364038"
format_stamp: "Formatted at 2026-08-08 06:04:37 on dist-test-slave-zj8q"
server_key: "c2b9bad04d6d7d4a4525bd6c8947c322"
server_key_iv: "40215a783fb3a6b9d390ad5c6abc3293"
server_key_version: "encryptionkey@0"
I20260808 06:04:37.404484 3375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "00147b47a31b4ab5a2080efc0f364038"
format_stamp: "Formatted at 2026-08-08 06:04:37 on dist-test-slave-zj8q"
server_key: "c2b9bad04d6d7d4a4525bd6c8947c322"
server_key_iv: "40215a783fb3a6b9d390ad5c6abc3293"
server_key_version: "encryptionkey@0"
I20260808 06:04:37.408054 3375 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 06:04:37.410338 3395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:37.411510 3375 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:04:37.411614 3375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "00147b47a31b4ab5a2080efc0f364038"
format_stamp: "Formatted at 2026-08-08 06:04:37 on dist-test-slave-zj8q"
server_key: "c2b9bad04d6d7d4a4525bd6c8947c322"
server_key_iv: "40215a783fb3a6b9d390ad5c6abc3293"
server_key_version: "encryptionkey@0"
I20260808 06:04:37.411737 3375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:37.429361 3375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:37.432103 3375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:37.432263 3375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:37.439957 3375 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:39871
I20260808 06:04:37.440004 3497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:39871 every 8 connection(s)
I20260808 06:04:37.440861 3375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:04:37.442364 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 3375
I20260808 06:04:37.442461 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:04:37.442709 1630 external_mini_cluster.cc:1428] Setting key e89390fa674757606f0f9746a36de908
I20260808 06:04:37.443784 3499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:37.449448 3499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00147b47a31b4ab5a2080efc0f364038: Bootstrap starting.
Aug 08 06:04:37 dist-test-slave-zj8q krb5kdc[3351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:04:37.451895 3499 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00147b47a31b4ab5a2080efc0f364038: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:37.452898 3499 log.cc:824] T 00000000000000000000000000000000 P 00147b47a31b4ab5a2080efc0f364038: Log is configured to *not* fsync() on all Append() calls
I20260808 06:04:37.454989 3499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00147b47a31b4ab5a2080efc0f364038: No bootstrap required, opened a new log
I20260808 06:04:37.456858 3502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:37.444074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40508 (local address 127.1.151.190:39871)
0808 06:04:37.444515 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:37.444529 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:37.444564 (+ 35us) server_negotiation.cc:407] Connection header received
0808 06:04:37.445194 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:37.445210 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:37.445541 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:37.445847 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:37.446698 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:37.447687 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:37.448529 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:37.448999 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:37.449149 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:37.451754 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:37.451759 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:37.451772 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:37.451798 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:37.453439 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:37.453911 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:37.453915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:37.453920 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:37.454030 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:37.454376 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:37.454379 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:37.454381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:37.454782 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:37.454939 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:37.455627 (+ 688us) server_negotiation.cc:299] Negotiation successful
0808 06:04:37.455941 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":139,"threads_started":1}
I20260808 06:04:37.458688 3499 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00147b47a31b4ab5a2080efc0f364038 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00147b47a31b4ab5a2080efc0f364038" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39871 } }
I20260808 06:04:37.458906 3499 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00147b47a31b4ab5a2080efc0f364038 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:37.458971 3499 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00147b47a31b4ab5a2080efc0f364038 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00147b47a31b4ab5a2080efc0f364038, State: Initialized, Role: FOLLOWER
I20260808 06:04:37.459367 3499 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00147b47a31b4ab5a2080efc0f364038 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00147b47a31b4ab5a2080efc0f364038" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39871 } }
I20260808 06:04:37.459491 3499 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00147b47a31b4ab5a2080efc0f364038 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:04:37.459545 3499 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00147b47a31b4ab5a2080efc0f364038 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:04:37.459638 3499 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00147b47a31b4ab5a2080efc0f364038 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:37.460446 3499 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00147b47a31b4ab5a2080efc0f364038 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00147b47a31b4ab5a2080efc0f364038" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39871 } }
I20260808 06:04:37.460750 3499 leader_election.cc:302] T 00000000000000000000000000000000 P 00147b47a31b4ab5a2080efc0f364038 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00147b47a31b4ab5a2080efc0f364038; no voters:
I20260808 06:04:37.461069 3499 leader_election.cc:288] T 00000000000000000000000000000000 P 00147b47a31b4ab5a2080efc0f364038 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:04:37.461148 3504 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00147b47a31b4ab5a2080efc0f364038 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:04:37.461370 3504 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00147b47a31b4ab5a2080efc0f364038 [term 1 LEADER]: Becoming Leader. State: Replica: 00147b47a31b4ab5a2080efc0f364038, State: Running, Role: LEADER
I20260808 06:04:37.461630 3504 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00147b47a31b4ab5a2080efc0f364038 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00147b47a31b4ab5a2080efc0f364038" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39871 } }
I20260808 06:04:37.462205 3499 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00147b47a31b4ab5a2080efc0f364038 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:04:37.463178 3505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00147b47a31b4ab5a2080efc0f364038 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00147b47a31b4ab5a2080efc0f364038" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00147b47a31b4ab5a2080efc0f364038" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39871 } } }
I20260808 06:04:37.463306 3505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00147b47a31b4ab5a2080efc0f364038 [sys.catalog]: This master's current role is: LEADER
I20260808 06:04:37.463573 3507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00147b47a31b4ab5a2080efc0f364038 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00147b47a31b4ab5a2080efc0f364038. Latest consensus state: current_term: 1 leader_uuid: "00147b47a31b4ab5a2080efc0f364038" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00147b47a31b4ab5a2080efc0f364038" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39871 } } }
I20260808 06:04:37.463675 3507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00147b47a31b4ab5a2080efc0f364038 [sys.catalog]: This master's current role is: LEADER
I20260808 06:04:37.463747 3527 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:04:37.466925 3527 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:04:37.472303 3527 catalog_manager.cc:1347] Generated new cluster ID: f3b2714bea984af0a3f86c671c6d8d53
I20260808 06:04:37.472363 3527 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:04:37.489168 3527 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:04:37.490406 3527 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:04:37.512442 3527 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00147b47a31b4ab5a2080efc0f364038: Generated new TSK 0
I20260808 06:04:37.513141 3527 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.151.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab.
Entry for principal kudu/127.1.151.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab.
I20260808 06:04:37.544341 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.129:0
--local_ip_for_outbound_sockets=127.1.151.129
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:39871
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:35451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:04:37.680100 3543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:37.680337 3543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:37.680377 3543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:37.684193 3543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:37.684244 3543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:37.684279 3543 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 06:04:37.684309 3543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.129
I20260808 06:04:37.689275 3543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:35451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.151.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.151.190:39871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.129
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:37.690212 3543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:37.691207 3543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:37.700837 3551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:37.700852 3550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:37.700837 3553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:37.701089 3543 server_base.cc:1034] running on GCE node
I20260808 06:04:37.701546 3543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:37.702189 3543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:37.703352 3543 hybrid_clock.cc:648] HybridClock initialized: now 1786169077703328 us; error 41 us; skew 500 ppm
Aug 08 06:04:37 dist-test-slave-zj8q krb5kdc[3351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169077, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:04:37.706003 3543 init.cc:377] Logged in from keytab as kudu/127.1.151.129@KRBTEST.COM (short username kudu)
I20260808 06:04:37.706909 3543 webserver.cc:460] Webserver started at http://127.1.151.129:41041/ using document root <none> and password file <none>
I20260808 06:04:37.707418 3543 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:37.707474 3543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:37.707640 3543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:37.709560 3543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-0/data/instance:
uuid: "830588ce3f974ac193a24845e0f79841"
format_stamp: "Formatted at 2026-08-08 06:04:37 on dist-test-slave-zj8q"
server_key: "a0d94846aee0f8570f1958b43b847269"
server_key_iv: "ff7f4b4b219ac6de6ed7bbc26b60f19b"
server_key_version: "encryptionkey@0"
I20260808 06:04:37.710076 3543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance:
uuid: "830588ce3f974ac193a24845e0f79841"
format_stamp: "Formatted at 2026-08-08 06:04:37 on dist-test-slave-zj8q"
server_key: "a0d94846aee0f8570f1958b43b847269"
server_key_iv: "ff7f4b4b219ac6de6ed7bbc26b60f19b"
server_key_version: "encryptionkey@0"
I20260808 06:04:37.713581 3543 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 06:04:37.715906 3564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:37.717007 3543 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260808 06:04:37.717120 3543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-0/wal
uuid: "830588ce3f974ac193a24845e0f79841"
format_stamp: "Formatted at 2026-08-08 06:04:37 on dist-test-slave-zj8q"
server_key: "a0d94846aee0f8570f1958b43b847269"
server_key_iv: "ff7f4b4b219ac6de6ed7bbc26b60f19b"
server_key_version: "encryptionkey@0"
I20260808 06:04:37.717234 3543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:37.733701 3543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:37.736801 3543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:37.736970 3543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:37.738584 3543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:04:37.738636 3543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:37.738682 3543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:04:37.738710 3543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 06:04:37 dist-test-slave-zj8q krb5kdc[3351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169077, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:04:37.749111 3543 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.129:39609
I20260808 06:04:37.749806 3736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.129:39609 every 8 connection(s)
I20260808 06:04:37.749989 3543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
I20260808 06:04:37.751241 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 3543
I20260808 06:04:37.751360 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance
I20260808 06:04:37.751664 1630 external_mini_cluster.cc:1428] Setting key 8af3626c84cad27d2533729e11ae5843
I20260808 06:04:37.752643 3580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:37.739686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39871 (local address 127.1.151.129:40193)
0808 06:04:37.740175 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:37.740233 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:37.741289 (+ 1056us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:37.741935 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:37.741944 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:37.742310 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:37.742839 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:37.742859 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:37.743678 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:37.743682 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:37.744680 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:37.744690 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:37.745192 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:37.745197 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:37.745379 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:37.745979 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:04:37.745999 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:37.747951 (+ 1952us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:37.749709 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:37.749715 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:37.749728 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:37.750052 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:37.750346 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:37.750349 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:37.750351 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:37.750527 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:37.750970 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:37.750976 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:37.751257 (+ 281us) client_negotiation.cc:769] Sending connection context
0808 06:04:37.751458 (+ 201us) client_negotiation.cc:240] Negotiation successful
0808 06:04:37.751781 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":117,"threads_started":1}
I20260808 06:04:37.753274 3502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:37.739331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:40193 (local address 127.1.151.190:39871)
0808 06:04:37.739490 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:37.739495 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:37.740352 (+ 857us) server_negotiation.cc:407] Connection header received
0808 06:04:37.741571 (+ 1219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:37.741575 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:37.741636 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:37.741728 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:37.742997 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:37.743553 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:37.744803 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:37.745035 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:37.745136 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:37.748141 (+ 3005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:37.748146 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:37.748149 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:37.748178 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:37.749577 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:37.750142 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:37.750145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:37.750146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:37.750254 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:37.750657 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:37.750661 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:37.750663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:37.750871 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:37.750953 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:37.752994 (+ 2041us) server_negotiation.cc:299] Negotiation successful
0808 06:04:37.753126 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 06:04:37.754313 3738 heartbeater.cc:347] Connected to a master server at 127.1.151.190:39871
I20260808 06:04:37.754441 3738 heartbeater.cc:464] Registering TS with master...
I20260808 06:04:37.754935 3738 heartbeater.cc:511] Master 127.1.151.190:39871 requested a full tablet report, sending...
I20260808 06:04:37.756439 3418 ts_manager.cc:194] Registered new tserver with Master: 830588ce3f974ac193a24845e0f79841 (127.1.151.129:39609)
I20260808 06:04:37.757535 3418 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.129@KRBTEST.COM'} at 127.1.151.129:40193
I20260808 06:04:37.764583 3749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:37.756333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39871 (local address 127.1.151.129:57903)
0808 06:04:37.756626 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:37.756650 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:37.756738 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:37.758417 (+ 1679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:37.758420 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:37.758580 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:37.758688 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:37.758694 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:37.759597 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:37.759599 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:37.760160 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:37.760167 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:37.760684 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:37.760686 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:37.760751 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:37.761168 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:04:37.761179 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:37.761460 (+ 281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:37.763265 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:37.763268 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:37.763269 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:37.763461 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:37.763757 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:37.763760 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:37.763762 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:37.763835 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:37.764201 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:37.764203 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:37.764275 (+ 72us) client_negotiation.cc:769] Sending connection context
0808 06:04:37.764347 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 06:04:37.764451 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":119,"threads_started":1}
I20260808 06:04:37.764684 3502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:37.758096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:57903 (local address 127.1.151.190:39871)
0808 06:04:37.758219 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:37.758223 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:37.758236 (+ 13us) server_negotiation.cc:407] Connection header received
0808 06:04:37.758268 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:37.758271 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:37.758307 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:37.758378 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:37.758796 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:37.759496 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:37.760268 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:37.760569 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:37.760671 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:37.761590 (+ 919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:37.761595 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:37.761598 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:37.761712 (+ 114us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:37.763164 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:37.763562 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:37.763565 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:37.763567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:37.763663 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:37.763914 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:37.763917 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:37.763919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:37.764120 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:37.764220 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:37.764419 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 06:04:37.764529 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
WARNING: no policy specified for kudu/127.1.151.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab.
Entry for principal kudu/127.1.151.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab.
I20260808 06:04:37.782191 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.130:0
--local_ip_for_outbound_sockets=127.1.151.130
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:39871
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:35451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:04:37.890990 3751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:37.891230 3751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:37.891270 3751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:37.895072 3751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:37.895118 3751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:37.895154 3751 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 06:04:37.895184 3751 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.130
I20260808 06:04:37.900164 3751 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:35451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.151.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.151.190:39871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.130
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:37.901036 3751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:37.901947 3751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:37.910863 3764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:37.910974 3757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:37.910861 3759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:37.911067 3751 server_base.cc:1034] running on GCE node
I20260808 06:04:37.911535 3751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:37.912122 3751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:37.913293 3751 hybrid_clock.cc:648] HybridClock initialized: now 1786169077913277 us; error 24 us; skew 500 ppm
Aug 08 06:04:37 dist-test-slave-zj8q krb5kdc[3351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169077, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:04:37.916044 3751 init.cc:377] Logged in from keytab as kudu/127.1.151.130@KRBTEST.COM (short username kudu)
I20260808 06:04:37.916826 3751 webserver.cc:460] Webserver started at http://127.1.151.130:40377/ using document root <none> and password file <none>
I20260808 06:04:37.917335 3751 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:37.917420 3751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:37.917660 3751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:37.919653 3751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-1/data/instance:
uuid: "7fd2f1f56a9143fcb30708ea6a309094"
format_stamp: "Formatted at 2026-08-08 06:04:37 on dist-test-slave-zj8q"
server_key: "7a248816b0944be587286163c3a31d92"
server_key_iv: "020c9a5c5a53cc1bddb6c046ccd8fca3"
server_key_version: "encryptionkey@0"
I20260808 06:04:37.920210 3751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance:
uuid: "7fd2f1f56a9143fcb30708ea6a309094"
format_stamp: "Formatted at 2026-08-08 06:04:37 on dist-test-slave-zj8q"
server_key: "7a248816b0944be587286163c3a31d92"
server_key_iv: "020c9a5c5a53cc1bddb6c046ccd8fca3"
server_key_version: "encryptionkey@0"
I20260808 06:04:37.923837 3751 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 06:04:37.926124 3774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:37.927335 3751 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:04:37.927444 3751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-1/wal
uuid: "7fd2f1f56a9143fcb30708ea6a309094"
format_stamp: "Formatted at 2026-08-08 06:04:37 on dist-test-slave-zj8q"
server_key: "7a248816b0944be587286163c3a31d92"
server_key_iv: "020c9a5c5a53cc1bddb6c046ccd8fca3"
server_key_version: "encryptionkey@0"
I20260808 06:04:37.927531 3751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:37.945941 3751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:37.948863 3751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:37.949035 3751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:37.950407 3751 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:04:37.950455 3751 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:37.950506 3751 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:04:37.950533 3751 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 06:04:37 dist-test-slave-zj8q krb5kdc[3351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169077, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:04:37.960690 3751 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.130:44875
I20260808 06:04:37.960909 3957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.130:44875 every 8 connection(s)
I20260808 06:04:37.961615 3751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
I20260808 06:04:37.964613 3502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:37.951138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:32995 (local address 127.1.151.190:39871)
0808 06:04:37.951283 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:37.951288 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:37.952473 (+ 1185us) server_negotiation.cc:407] Connection header received
0808 06:04:37.953532 (+ 1059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:37.953535 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:37.953593 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:37.953700 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:37.955448 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:37.956274 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:37.957939 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:37.958187 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:37.958274 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:37.961180 (+ 2906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:37.961184 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:37.961187 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:37.961216 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:37.962717 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:37.963303 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:37.963306 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:37.963307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:37.963386 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:37.963676 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:37.963678 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:37.963679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:37.963849 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:37.963959 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:37.964397 (+ 438us) server_negotiation.cc:299] Negotiation successful
0808 06:04:37.964493 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 06:04:37.965350 3786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:37.951885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39871 (local address 127.1.151.130:32995)
0808 06:04:37.952333 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:37.952379 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:37.953307 (+ 928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:37.954041 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:37.954049 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:37.954597 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:37.955281 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:37.955306 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:37.956430 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:37.956434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:37.957822 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:37.957835 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:37.958345 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:37.958350 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:37.958549 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:37.959159 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:04:37.959181 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:37.961012 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:37.962839 (+ 1827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:37.962844 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:37.962857 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:37.963219 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:37.963455 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:37.963457 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:37.963458 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:37.963598 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:37.963966 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:37.963972 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:37.964196 (+ 224us) client_negotiation.cc:769] Sending connection context
0808 06:04:37.964342 (+ 146us) client_negotiation.cc:240] Negotiation successful
0808 06:04:37.964569 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":105,"threads_started":1}
I20260808 06:04:37.966562 3958 heartbeater.cc:347] Connected to a master server at 127.1.151.190:39871
I20260808 06:04:37.966704 3958 heartbeater.cc:464] Registering TS with master...
I20260808 06:04:37.967154 3958 heartbeater.cc:511] Master 127.1.151.190:39871 requested a full tablet report, sending...
I20260808 06:04:37.968152 3419 ts_manager.cc:194] Registered new tserver with Master: 7fd2f1f56a9143fcb30708ea6a309094 (127.1.151.130:44875)
I20260808 06:04:37.968171 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 3751
I20260808 06:04:37.968272 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance
I20260808 06:04:37.968575 1630 external_mini_cluster.cc:1428] Setting key 500ea23c9abe61cfad024b49e98937b8
I20260808 06:04:37.968850 3419 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.130@KRBTEST.COM'} at 127.1.151.130:32995
I20260808 06:04:37.975271 3967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:37.968573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39871 (local address 127.1.151.130:34519)
0808 06:04:37.968849 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:37.968864 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:37.969002 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:37.969301 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:37.969304 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:37.969486 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:37.969591 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:37.969598 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:37.970372 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:37.970375 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:37.970921 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:37.970927 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:37.971441 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:37.971444 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:37.971525 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:37.971949 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:04:37.971960 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:37.972283 (+ 323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:37.973832 (+ 1549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:37.973835 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:37.973837 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:37.974073 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:37.974353 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:37.974355 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:37.974357 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:37.974431 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:37.974882 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:37.974884 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:37.974955 (+ 71us) client_negotiation.cc:769] Sending connection context
0808 06:04:37.975027 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 06:04:37.975130 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":101,"threads_started":1}
I20260808 06:04:37.975351 3502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:37.968983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:34519 (local address 127.1.151.190:39871)
0808 06:04:37.969118 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:37.969122 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:37.969133 (+ 11us) server_negotiation.cc:407] Connection header received
0808 06:04:37.969162 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:37.969165 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:37.969207 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:37.969280 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:37.969713 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:37.970241 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:37.971030 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:37.971307 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:37.971406 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:37.972416 (+ 1010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:37.972420 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:37.972422 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:37.972447 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:37.973721 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:37.974186 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:37.974189 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:37.974191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:37.974272 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:37.974505 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:37.974508 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:37.974509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:37.974794 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:37.974885 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:37.975096 (+ 211us) server_negotiation.cc:299] Negotiation successful
0808 06:04:37.975207 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
WARNING: no policy specified for kudu/127.1.151.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab.
Entry for principal kudu/127.1.151.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab.
I20260808 06:04:37.994526 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.131:0
--local_ip_for_outbound_sockets=127.1.151.131
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:39871
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:35451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:04:38.104856 3979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:38.105082 3979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:38.105131 3979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:38.109002 3979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:38.109050 3979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:38.109084 3979 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 06:04:38.109129 3979 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.131
I20260808 06:04:38.114012 3979 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:35451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.151.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.151.190:39871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.131
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:38.114957 3979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:38.115908 3979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:38.125026 3996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:38.125079 3993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:38.125198 3979 server_base.cc:1034] running on GCE node
W20260808 06:04:38.125046 3992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:38.125728 3979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:38.126369 3979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:38.127519 3979 hybrid_clock.cc:648] HybridClock initialized: now 1786169078127507 us; error 25 us; skew 500 ppm
Aug 08 06:04:38 dist-test-slave-zj8q krb5kdc[3351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169078, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:04:38.130123 3979 init.cc:377] Logged in from keytab as kudu/127.1.151.131@KRBTEST.COM (short username kudu)
I20260808 06:04:38.131011 3979 webserver.cc:460] Webserver started at http://127.1.151.131:40693/ using document root <none> and password file <none>
I20260808 06:04:38.131564 3979 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:38.131628 3979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:38.131793 3979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:38.133756 3979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-2/data/instance:
uuid: "009c1b5a8ce94ce79613d0a5473b4eb0"
format_stamp: "Formatted at 2026-08-08 06:04:38 on dist-test-slave-zj8q"
server_key: "f03ad6db5402c5dcc27ea55db08c900c"
server_key_iv: "1d9d39491d9cedcac4ca7a45bd2e9fc0"
server_key_version: "encryptionkey@0"
I20260808 06:04:38.134424 3979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance:
uuid: "009c1b5a8ce94ce79613d0a5473b4eb0"
format_stamp: "Formatted at 2026-08-08 06:04:38 on dist-test-slave-zj8q"
server_key: "f03ad6db5402c5dcc27ea55db08c900c"
server_key_iv: "1d9d39491d9cedcac4ca7a45bd2e9fc0"
server_key_version: "encryptionkey@0"
I20260808 06:04:38.138087 3979 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 06:04:38.140494 4003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:38.141664 3979 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:04:38.141762 3979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-2/wal
uuid: "009c1b5a8ce94ce79613d0a5473b4eb0"
format_stamp: "Formatted at 2026-08-08 06:04:38 on dist-test-slave-zj8q"
server_key: "f03ad6db5402c5dcc27ea55db08c900c"
server_key_iv: "1d9d39491d9cedcac4ca7a45bd2e9fc0"
server_key_version: "encryptionkey@0"
I20260808 06:04:38.141857 3979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:38.179324 3979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:38.182297 3979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:38.182461 3979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:38.183926 3979 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:04:38.183979 3979 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:38.184033 3979 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:04:38.184062 3979 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 06:04:38 dist-test-slave-zj8q krb5kdc[3351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169078, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:04:38.194293 3979 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.131:43093
I20260808 06:04:38.194377 4186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.131:43093 every 8 connection(s)
I20260808 06:04:38.195407 3979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
I20260808 06:04:38.197754 3502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:38.186326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:52955 (local address 127.1.151.190:39871)
0808 06:04:38.186459 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:38.186463 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:38.186480 (+ 17us) server_negotiation.cc:407] Connection header received
0808 06:04:38.186948 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:38.186952 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:38.187011 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:38.187066 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:38.188385 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.189205 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.190608 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.190954 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.191089 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:38.194231 (+ 3142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:38.194235 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:38.194238 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:38.194269 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:38.195760 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:38.196348 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:38.196352 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:38.196354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:38.196456 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:38.196808 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:38.196812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:38.196814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:38.197003 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:38.197092 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:38.197522 (+ 430us) server_negotiation.cc:299] Negotiation successful
0808 06:04:38.197634 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 06:04:38.198580 4019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:38.185267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39871 (local address 127.1.151.131:52955)
0808 06:04:38.185763 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:38.185805 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:38.186704 (+ 899us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:38.187427 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:38.187433 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:38.187747 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:38.188219 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:38.188238 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.189343 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.189346 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:38.190474 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:38.190485 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.191147 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.191152 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:38.191353 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:38.192481 (+ 1128us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:04:38.192497 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:38.194023 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:38.195870 (+ 1847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:38.195877 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:38.195891 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:38.196218 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:38.196537 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:38.196540 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:38.196541 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:38.196691 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:38.197094 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:38.197099 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:38.197327 (+ 228us) client_negotiation.cc:769] Sending connection context
0808 06:04:38.197476 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 06:04:38.197717 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":150,"threads_started":1}
I20260808 06:04:38.199836 4189 heartbeater.cc:347] Connected to a master server at 127.1.151.190:39871
I20260808 06:04:38.199980 4189 heartbeater.cc:464] Registering TS with master...
I20260808 06:04:38.200212 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 3979
I20260808 06:04:38.200320 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance
I20260808 06:04:38.200517 4189 heartbeater.cc:511] Master 127.1.151.190:39871 requested a full tablet report, sending...
I20260808 06:04:38.200588 1630 external_mini_cluster.cc:1428] Setting key da10fcf17e28eff6e8548f779aa6ba26
I20260808 06:04:38.201552 3419 ts_manager.cc:194] Registered new tserver with Master: 009c1b5a8ce94ce79613d0a5473b4eb0 (127.1.151.131:43093)
I20260808 06:04:38.202273 3419 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.131@KRBTEST.COM'} at 127.1.151.131:52955
I20260808 06:04:38.202725 1630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 06:04:38.208853 3502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:38.202082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:45497 (local address 127.1.151.190:39871)
0808 06:04:38.202208 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:38.202212 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:38.202779 (+ 567us) server_negotiation.cc:407] Connection header received
0808 06:04:38.202941 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:38.202944 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:38.202987 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:38.203046 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:38.203474 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.203968 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.204741 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.204952 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.205036 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:38.206033 (+ 997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:38.206036 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:38.206039 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:38.206061 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:38.207359 (+ 1298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:38.207756 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:38.207759 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:38.207761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:38.207839 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:38.208084 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:38.208087 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:38.208089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:38.208259 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:38.208343 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:38.208592 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 06:04:38.208709 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 06:04:38.208861 4202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:38.202315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39871 (local address 127.1.151.131:45497)
0808 06:04:38.202688 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:38.202701 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:38.202853 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:38.203075 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:38.203077 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:38.203276 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:38.203369 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:38.203375 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.204090 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.204093 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:38.204635 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:38.204644 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.205051 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.205054 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:38.205151 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:38.205605 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:04:38.205615 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:38.205907 (+ 292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:38.207464 (+ 1557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:38.207467 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:38.207470 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:38.207665 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:38.207919 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:38.207923 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:38.207925 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:38.208004 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:38.208350 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:38.208353 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:38.208441 (+ 88us) client_negotiation.cc:769] Sending connection context
0808 06:04:38.208528 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 06:04:38.208666 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":114,"threads_started":1}
Aug 08 06:04:38 dist-test-slave-zj8q krb5kdc[3351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169078, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 06:04:38 dist-test-slave-zj8q krb5kdc[3351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169078, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:04:38.229467 3502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:38.221767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40514 (local address 127.1.151.190:39871)
0808 06:04:38.221874 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:38.221877 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:38.221891 (+ 14us) server_negotiation.cc:407] Connection header received
0808 06:04:38.221927 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:38.221929 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:38.221963 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:38.222026 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:38.222767 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.223266 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.224130 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.224331 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.224402 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:38.226724 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:38.226727 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:38.226729 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:38.226751 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:38.227986 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:38.228375 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:38.228378 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:38.228379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:38.228452 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:38.228700 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:38.228703 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:38.228705 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:38.228870 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:38.228987 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:38.229216 (+ 229us) server_negotiation.cc:299] Negotiation successful
0808 06:04:38.229320 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 06:04:38.232352 3419 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40514:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 06:04:38.235342 3419 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 06:04:38.249318 4223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:38.243641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:44875 (local address 127.0.0.1:37740)
0808 06:04:38.243980 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:38.244048 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:38.244180 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:38.244990 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:38.244996 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:38.245033 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:04:38.245193 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:38.245201 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.246604 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.246607 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:38.247948 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:38.247959 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.248804 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.248808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:38.248908 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:38.248953 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 06:04:38.249053 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 06:04:38.249188 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":71,"threads_started":1}
I20260808 06:04:38.249534 4225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:38.243941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43704 (local address 127.1.151.131:43093)
0808 06:04:38.244388 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:38.244400 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:38.244630 (+ 230us) server_negotiation.cc:407] Connection header received
0808 06:04:38.244861 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:38.244867 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:38.245066 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:38.245224 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:04:38.245695 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.246723 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.248096 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.248882 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.248978 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:38.249128 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:38.249307 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 06:04:38.249401 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":107,"threads_started":1}
I20260808 06:04:38.249830 4227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:38.243633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:43093 (local address 127.0.0.1:43704)
0808 06:04:38.244530 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:38.244543 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:38.244708 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:38.245454 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:38.245457 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:38.245476 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:04:38.245579 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:38.245584 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.246837 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.246841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:38.247981 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:38.247988 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.249068 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.249072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:38.249165 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:38.249190 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 06:04:38.249267 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 06:04:38.249336 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":829,"mutex_wait_us":625,"thread_start_us":146,"threads_started":1}
I20260808 06:04:38.249871 4226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:38.243933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37740 (local address 127.1.151.130:44875)
0808 06:04:38.244479 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:38.244492 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:38.244518 (+ 26us) server_negotiation.cc:407] Connection header received
0808 06:04:38.244599 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:38.244604 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:38.244812 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:38.245352 (+ 540us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:04:38.245487 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.246489 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.248121 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.248664 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.248801 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:38.248971 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:38.249679 (+ 708us) server_negotiation.cc:299] Negotiation successful
0808 06:04:38.249764 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":140,"threads_started":1}
I20260808 06:04:38.251111 4232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:38.245272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:39609 (local address 127.0.0.1:45022)
0808 06:04:38.245571 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:38.245585 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:38.245736 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:38.246331 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:38.246334 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:38.246363 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:04:38.246467 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:38.246472 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.247572 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.247575 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:38.248627 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:38.248639 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.250691 (+ 2052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.250695 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:38.250797 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:38.250831 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 06:04:38.250907 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 06:04:38.250978 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":102,"threads_started":1}
I20260808 06:04:38.251221 4230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:38.244966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45022 (local address 127.1.151.129:39609)
0808 06:04:38.245318 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:38.245334 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:38.245667 (+ 333us) server_negotiation.cc:407] Connection header received
0808 06:04:38.245898 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:38.245903 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:38.246076 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:38.246250 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:04:38.246591 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.247453 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.248765 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.249576 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.249687 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:38.249832 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:38.250973 (+ 1141us) server_negotiation.cc:299] Negotiation successful
0808 06:04:38.251076 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":89,"threads_started":1}
I20260808 06:04:38.252756 3858 tablet_service.cc:1467] Processing CreateTablet for tablet 64c4b94cfc814833938c108f5538ad1b (DEFAULT_TABLE table=test-table [id=813fc757222e47b896ecf435ae7480e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:04:38.253202 4078 tablet_service.cc:1467] Processing CreateTablet for tablet 64c4b94cfc814833938c108f5538ad1b (DEFAULT_TABLE table=test-table [id=813fc757222e47b896ecf435ae7480e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:04:38.253789 3858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64c4b94cfc814833938c108f5538ad1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:38.253860 3647 tablet_service.cc:1467] Processing CreateTablet for tablet 64c4b94cfc814833938c108f5538ad1b (DEFAULT_TABLE table=test-table [id=813fc757222e47b896ecf435ae7480e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:04:38.254283 4078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64c4b94cfc814833938c108f5538ad1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:38.254853 3647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64c4b94cfc814833938c108f5538ad1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:38.260356 4233 tablet_bootstrap.cc:492] T 64c4b94cfc814833938c108f5538ad1b P 7fd2f1f56a9143fcb30708ea6a309094: Bootstrap starting.
I20260808 06:04:38.261852 4235 tablet_bootstrap.cc:492] T 64c4b94cfc814833938c108f5538ad1b P 830588ce3f974ac193a24845e0f79841: Bootstrap starting.
I20260808 06:04:38.262483 4234 tablet_bootstrap.cc:492] T 64c4b94cfc814833938c108f5538ad1b P 009c1b5a8ce94ce79613d0a5473b4eb0: Bootstrap starting.
I20260808 06:04:38.262992 4233 tablet_bootstrap.cc:654] T 64c4b94cfc814833938c108f5538ad1b P 7fd2f1f56a9143fcb30708ea6a309094: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:38.263942 4233 log.cc:824] T 64c4b94cfc814833938c108f5538ad1b P 7fd2f1f56a9143fcb30708ea6a309094: Log is configured to *not* fsync() on all Append() calls
I20260808 06:04:38.264418 4234 tablet_bootstrap.cc:654] T 64c4b94cfc814833938c108f5538ad1b P 009c1b5a8ce94ce79613d0a5473b4eb0: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:38.264501 4235 tablet_bootstrap.cc:654] T 64c4b94cfc814833938c108f5538ad1b P 830588ce3f974ac193a24845e0f79841: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:38.265203 4234 log.cc:824] T 64c4b94cfc814833938c108f5538ad1b P 009c1b5a8ce94ce79613d0a5473b4eb0: Log is configured to *not* fsync() on all Append() calls
I20260808 06:04:38.265277 4235 log.cc:824] T 64c4b94cfc814833938c108f5538ad1b P 830588ce3f974ac193a24845e0f79841: Log is configured to *not* fsync() on all Append() calls
I20260808 06:04:38.266052 4233 tablet_bootstrap.cc:492] T 64c4b94cfc814833938c108f5538ad1b P 7fd2f1f56a9143fcb30708ea6a309094: No bootstrap required, opened a new log
I20260808 06:04:38.266278 4233 ts_tablet_manager.cc:1397] T 64c4b94cfc814833938c108f5538ad1b P 7fd2f1f56a9143fcb30708ea6a309094: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260808 06:04:38.266948 4235 tablet_bootstrap.cc:492] T 64c4b94cfc814833938c108f5538ad1b P 830588ce3f974ac193a24845e0f79841: No bootstrap required, opened a new log
I20260808 06:04:38.267130 4235 ts_tablet_manager.cc:1397] T 64c4b94cfc814833938c108f5538ad1b P 830588ce3f974ac193a24845e0f79841: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 06:04:38.267153 4234 tablet_bootstrap.cc:492] T 64c4b94cfc814833938c108f5538ad1b P 009c1b5a8ce94ce79613d0a5473b4eb0: No bootstrap required, opened a new log
I20260808 06:04:38.267365 4234 ts_tablet_manager.cc:1397] T 64c4b94cfc814833938c108f5538ad1b P 009c1b5a8ce94ce79613d0a5473b4eb0: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 06:04:38.270201 4235 raft_consensus.cc:348] T 64c4b94cfc814833938c108f5538ad1b P 830588ce3f974ac193a24845e0f79841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } } peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } }
I20260808 06:04:38.270190 4233 raft_consensus.cc:348] T 64c4b94cfc814833938c108f5538ad1b P 7fd2f1f56a9143fcb30708ea6a309094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } } peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } }
I20260808 06:04:38.270427 4235 raft_consensus.cc:374] T 64c4b94cfc814833938c108f5538ad1b P 830588ce3f974ac193a24845e0f79841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:38.270439 4233 raft_consensus.cc:374] T 64c4b94cfc814833938c108f5538ad1b P 7fd2f1f56a9143fcb30708ea6a309094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:38.270483 4233 raft_consensus.cc:729] T 64c4b94cfc814833938c108f5538ad1b P 7fd2f1f56a9143fcb30708ea6a309094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd2f1f56a9143fcb30708ea6a309094, State: Initialized, Role: FOLLOWER
I20260808 06:04:38.270489 4235 raft_consensus.cc:729] T 64c4b94cfc814833938c108f5538ad1b P 830588ce3f974ac193a24845e0f79841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 830588ce3f974ac193a24845e0f79841, State: Initialized, Role: FOLLOWER
I20260808 06:04:38.270900 4235 consensus_queue.cc:260] T 64c4b94cfc814833938c108f5538ad1b P 830588ce3f974ac193a24845e0f79841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } } peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } }
I20260808 06:04:38.270908 4233 consensus_queue.cc:260] T 64c4b94cfc814833938c108f5538ad1b P 7fd2f1f56a9143fcb30708ea6a309094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } } peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } }
I20260808 06:04:38.270959 4234 raft_consensus.cc:348] T 64c4b94cfc814833938c108f5538ad1b P 009c1b5a8ce94ce79613d0a5473b4eb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } } peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } }
I20260808 06:04:38.271198 4234 raft_consensus.cc:374] T 64c4b94cfc814833938c108f5538ad1b P 009c1b5a8ce94ce79613d0a5473b4eb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:38.271260 4234 raft_consensus.cc:729] T 64c4b94cfc814833938c108f5538ad1b P 009c1b5a8ce94ce79613d0a5473b4eb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 009c1b5a8ce94ce79613d0a5473b4eb0, State: Initialized, Role: FOLLOWER
I20260808 06:04:38.271646 3738 heartbeater.cc:503] Master 127.1.151.190:39871 was elected leader, sending a full tablet report...
I20260808 06:04:38.271621 4234 consensus_queue.cc:260] T 64c4b94cfc814833938c108f5538ad1b P 009c1b5a8ce94ce79613d0a5473b4eb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } } peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } }
I20260808 06:04:38.271836 4235 ts_tablet_manager.cc:1428] T 64c4b94cfc814833938c108f5538ad1b P 830588ce3f974ac193a24845e0f79841: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 06:04:38.271842 4233 ts_tablet_manager.cc:1428] T 64c4b94cfc814833938c108f5538ad1b P 7fd2f1f56a9143fcb30708ea6a309094: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 06:04:38.272161 3958 heartbeater.cc:503] Master 127.1.151.190:39871 was elected leader, sending a full tablet report...
I20260808 06:04:38.272472 4234 ts_tablet_manager.cc:1428] T 64c4b94cfc814833938c108f5538ad1b P 009c1b5a8ce94ce79613d0a5473b4eb0: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 06:04:38.272593 4189 heartbeater.cc:503] Master 127.1.151.190:39871 was elected leader, sending a full tablet report...
I20260808 06:04:38.310218 4243 raft_consensus.cc:482] T 64c4b94cfc814833938c108f5538ad1b P 7fd2f1f56a9143fcb30708ea6a309094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:04:38.310400 4243 raft_consensus.cc:504] T 64c4b94cfc814833938c108f5538ad1b P 7fd2f1f56a9143fcb30708ea6a309094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } } peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } }
I20260808 06:04:38.311447 4243 leader_election.cc:288] T 64c4b94cfc814833938c108f5538ad1b P 7fd2f1f56a9143fcb30708ea6a309094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 009c1b5a8ce94ce79613d0a5473b4eb0 (127.1.151.131:43093), 830588ce3f974ac193a24845e0f79841 (127.1.151.129:39609)
I20260808 06:04:38.314954 3786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:38.311701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:43093 (local address 127.1.151.130:40655)
0808 06:04:38.311837 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:38.311853 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:38.311939 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:38.312238 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:38.312241 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:38.312259 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:04:38.312361 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:38.312366 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.313095 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.313098 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:38.313880 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:38.313888 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.314546 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.314549 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:38.314665 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:38.314683 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 06:04:38.314748 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 06:04:38.314822 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 06:04:38.314954 4225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:38.311772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:40655 (local address 127.1.151.131:43093)
0808 06:04:38.311915 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:38.311920 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:38.311935 (+ 15us) server_negotiation.cc:407] Connection header received
0808 06:04:38.312033 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:38.312036 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:38.312084 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:38.312178 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:04:38.312468 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.312986 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.314004 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.314428 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.314542 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:38.314595 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:38.314776 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 06:04:38.314838 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 06:04:38.315588 4112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64c4b94cfc814833938c108f5538ad1b" candidate_uuid: "7fd2f1f56a9143fcb30708ea6a309094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" is_pre_election: true
I20260808 06:04:38.315912 4112 raft_consensus.cc:2454] T 64c4b94cfc814833938c108f5538ad1b P 009c1b5a8ce94ce79613d0a5473b4eb0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd2f1f56a9143fcb30708ea6a309094 in term 0.
I20260808 06:04:38.315990 4248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:38.311959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:39609 (local address 127.1.151.130:56359)
0808 06:04:38.312309 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:38.312320 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:38.312412 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:38.312612 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:38.312615 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:38.312630 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:04:38.312776 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:38.312781 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.313492 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.313495 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:38.314806 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:38.314817 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.315585 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.315589 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:38.315692 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:38.315715 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 06:04:38.315819 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 06:04:38.315888 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":127,"threads_started":1}
I20260808 06:04:38.316049 4230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:38.312081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:56359 (local address 127.1.151.129:39609)
0808 06:04:38.312216 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:38.312221 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:38.312391 (+ 170us) server_negotiation.cc:407] Connection header received
0808 06:04:38.312491 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:38.312495 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:38.312537 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:38.312598 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:04:38.312878 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.313390 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.314934 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.315454 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.315585 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:38.315636 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:38.315853 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 06:04:38.315937 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 06:04:38.316433 3777 leader_election.cc:302] T 64c4b94cfc814833938c108f5538ad1b P 7fd2f1f56a9143fcb30708ea6a309094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 009c1b5a8ce94ce79613d0a5473b4eb0, 7fd2f1f56a9143fcb30708ea6a309094; no voters:
I20260808 06:04:38.316628 3673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64c4b94cfc814833938c108f5538ad1b" candidate_uuid: "7fd2f1f56a9143fcb30708ea6a309094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "830588ce3f974ac193a24845e0f79841" is_pre_election: true
I20260808 06:04:38.316715 4243 raft_consensus.cc:2781] T 64c4b94cfc814833938c108f5538ad1b P 7fd2f1f56a9143fcb30708ea6a309094 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 06:04:38.316798 4243 raft_consensus.cc:482] T 64c4b94cfc814833938c108f5538ad1b P 7fd2f1f56a9143fcb30708ea6a309094 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 06:04:38.316852 4243 raft_consensus.cc:3037] T 64c4b94cfc814833938c108f5538ad1b P 7fd2f1f56a9143fcb30708ea6a309094 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:38.316942 3673 raft_consensus.cc:2454] T 64c4b94cfc814833938c108f5538ad1b P 830588ce3f974ac193a24845e0f79841 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd2f1f56a9143fcb30708ea6a309094 in term 0.
I20260808 06:04:38.317961 4243 raft_consensus.cc:504] T 64c4b94cfc814833938c108f5538ad1b P 7fd2f1f56a9143fcb30708ea6a309094 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } } peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } }
I20260808 06:04:38.318303 4243 leader_election.cc:288] T 64c4b94cfc814833938c108f5538ad1b P 7fd2f1f56a9143fcb30708ea6a309094 [CANDIDATE]: Term 1 election: Requested vote from peers 009c1b5a8ce94ce79613d0a5473b4eb0 (127.1.151.131:43093), 830588ce3f974ac193a24845e0f79841 (127.1.151.129:39609)
I20260808 06:04:38.318763 4112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64c4b94cfc814833938c108f5538ad1b" candidate_uuid: "7fd2f1f56a9143fcb30708ea6a309094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0"
I20260808 06:04:38.318809 3673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64c4b94cfc814833938c108f5538ad1b" candidate_uuid: "7fd2f1f56a9143fcb30708ea6a309094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "830588ce3f974ac193a24845e0f79841"
I20260808 06:04:38.318904 4112 raft_consensus.cc:3037] T 64c4b94cfc814833938c108f5538ad1b P 009c1b5a8ce94ce79613d0a5473b4eb0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:38.318928 3673 raft_consensus.cc:3037] T 64c4b94cfc814833938c108f5538ad1b P 830588ce3f974ac193a24845e0f79841 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:38.320210 3673 raft_consensus.cc:2454] T 64c4b94cfc814833938c108f5538ad1b P 830588ce3f974ac193a24845e0f79841 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd2f1f56a9143fcb30708ea6a309094 in term 1.
I20260808 06:04:38.320502 4112 raft_consensus.cc:2454] T 64c4b94cfc814833938c108f5538ad1b P 009c1b5a8ce94ce79613d0a5473b4eb0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd2f1f56a9143fcb30708ea6a309094 in term 1.
I20260808 06:04:38.320519 3777 leader_election.cc:302] T 64c4b94cfc814833938c108f5538ad1b P 7fd2f1f56a9143fcb30708ea6a309094 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd2f1f56a9143fcb30708ea6a309094, 830588ce3f974ac193a24845e0f79841; no voters:
I20260808 06:04:38.320685 4243 raft_consensus.cc:2781] T 64c4b94cfc814833938c108f5538ad1b P 7fd2f1f56a9143fcb30708ea6a309094 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:04:38.320932 4243 raft_consensus.cc:686] T 64c4b94cfc814833938c108f5538ad1b P 7fd2f1f56a9143fcb30708ea6a309094 [term 1 LEADER]: Becoming Leader. State: Replica: 7fd2f1f56a9143fcb30708ea6a309094, State: Running, Role: LEADER
I20260808 06:04:38.321297 4243 consensus_queue.cc:237] T 64c4b94cfc814833938c108f5538ad1b P 7fd2f1f56a9143fcb30708ea6a309094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } } peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } }
I20260808 06:04:38.324502 3414 catalog_manager.cc:5495] T 64c4b94cfc814833938c108f5538ad1b P 7fd2f1f56a9143fcb30708ea6a309094 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fd2f1f56a9143fcb30708ea6a309094 (127.1.151.130). New cstate: current_term: 1 leader_uuid: "7fd2f1f56a9143fcb30708ea6a309094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } health_report { overall_health: UNKNOWN } } }
I20260808 06:04:38.355033 4226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:38.351393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37742 (local address 127.1.151.130:44875)
0808 06:04:38.351518 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:38.351522 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:38.351537 (+ 15us) server_negotiation.cc:407] Connection header received
0808 06:04:38.351759 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:38.351762 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:38.351817 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:38.351899 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:04:38.352158 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.352670 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.353547 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.353770 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.353858 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:38.354209 (+ 351us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:04:38.354630 (+ 421us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:04:38.354740 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:38.354838 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 06:04:38.354897 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 06:04:38.360519 3673 raft_consensus.cc:1261] T 64c4b94cfc814833938c108f5538ad1b P 830588ce3f974ac193a24845e0f79841 [term 1 FOLLOWER]: Refusing update from remote peer 7fd2f1f56a9143fcb30708ea6a309094: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:04:38.360522 4112 raft_consensus.cc:1261] T 64c4b94cfc814833938c108f5538ad1b P 009c1b5a8ce94ce79613d0a5473b4eb0 [term 1 FOLLOWER]: Refusing update from remote peer 7fd2f1f56a9143fcb30708ea6a309094: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:04:38.361151 4243 consensus_queue.cc:1035] T 64c4b94cfc814833938c108f5538ad1b P 7fd2f1f56a9143fcb30708ea6a309094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:04:38.361387 4249 consensus_queue.cc:1035] T 64c4b94cfc814833938c108f5538ad1b P 7fd2f1f56a9143fcb30708ea6a309094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:04:38.369572 4256 mvcc.cc:205] Tried to move back new op lower bound from 7316148544957816832 to 7316148544806567936. Current Snapshot: MvccSnapshot[applied={T|T < 7316148544957816832}]
I20260808 06:04:38.379930 3417 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:04:38.380172 3417 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:04:38.382644 3417 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 00147b47a31b4ab5a2080efc0f364038: Sending DeleteTablet for 3 replicas of tablet 64c4b94cfc814833938c108f5538ad1b
I20260808 06:04:38.383389 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet 64c4b94cfc814833938c108f5538ad1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:04:38 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:37740
I20260808 06:04:38.383404 4078 tablet_service.cc:1514] Processing DeleteTablet for tablet 64c4b94cfc814833938c108f5538ad1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:04:38 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:43704
I20260808 06:04:38.383459 3647 tablet_service.cc:1514] Processing DeleteTablet for tablet 64c4b94cfc814833938c108f5538ad1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:04:38 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:45022
I20260808 06:04:38.384264 4270 tablet_replica.cc:330] T 64c4b94cfc814833938c108f5538ad1b P 830588ce3f974ac193a24845e0f79841: stopping tablet replica
I20260808 06:04:38.384508 3417 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40514:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 06:04:38.384851 3417 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 06:04:38.385546 4272 tablet_replica.cc:330] T 64c4b94cfc814833938c108f5538ad1b P 009c1b5a8ce94ce79613d0a5473b4eb0: stopping tablet replica
I20260808 06:04:38.385823 4272 raft_consensus.cc:2229] T 64c4b94cfc814833938c108f5538ad1b P 009c1b5a8ce94ce79613d0a5473b4eb0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:04:38.386497 4270 raft_consensus.cc:2229] T 64c4b94cfc814833938c108f5538ad1b P 830588ce3f974ac193a24845e0f79841 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:04:38.387027 4270 raft_consensus.cc:2258] T 64c4b94cfc814833938c108f5538ad1b P 830588ce3f974ac193a24845e0f79841 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:04:38.386121 4272 raft_consensus.cc:2258] T 64c4b94cfc814833938c108f5538ad1b P 009c1b5a8ce94ce79613d0a5473b4eb0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:04:38.389070 4270 ts_tablet_manager.cc:1905] T 64c4b94cfc814833938c108f5538ad1b P 830588ce3f974ac193a24845e0f79841: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:04:38.389561 3647 tablet_service.cc:1467] Processing CreateTablet for tablet 801601d4b0a04720b6aaa927e16971ac (DEFAULT_TABLE table=test-table [id=b26da4514c934426bfb8fa55f281213f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:04:38.389570 3858 tablet_service.cc:1467] Processing CreateTablet for tablet 801601d4b0a04720b6aaa927e16971ac (DEFAULT_TABLE table=test-table [id=b26da4514c934426bfb8fa55f281213f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:04:38.389720 4078 tablet_service.cc:1467] Processing CreateTablet for tablet 801601d4b0a04720b6aaa927e16971ac (DEFAULT_TABLE table=test-table [id=b26da4514c934426bfb8fa55f281213f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:04:38.389829 3858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 801601d4b0a04720b6aaa927e16971ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:38.389884 3647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 801601d4b0a04720b6aaa927e16971ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:38.389943 4078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 801601d4b0a04720b6aaa927e16971ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:38.392194 4234 tablet_bootstrap.cc:492] T 801601d4b0a04720b6aaa927e16971ac P 009c1b5a8ce94ce79613d0a5473b4eb0: Bootstrap starting.
I20260808 06:04:38.392259 4235 tablet_bootstrap.cc:492] T 801601d4b0a04720b6aaa927e16971ac P 830588ce3f974ac193a24845e0f79841: Bootstrap starting.
I20260808 06:04:38.393122 4234 tablet_bootstrap.cc:654] T 801601d4b0a04720b6aaa927e16971ac P 009c1b5a8ce94ce79613d0a5473b4eb0: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:38.393142 4235 tablet_bootstrap.cc:654] T 801601d4b0a04720b6aaa927e16971ac P 830588ce3f974ac193a24845e0f79841: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:38.394193 4233 tablet_bootstrap.cc:492] T 801601d4b0a04720b6aaa927e16971ac P 7fd2f1f56a9143fcb30708ea6a309094: Bootstrap starting.
I20260808 06:04:38.394259 4235 tablet_bootstrap.cc:492] T 801601d4b0a04720b6aaa927e16971ac P 830588ce3f974ac193a24845e0f79841: No bootstrap required, opened a new log
I20260808 06:04:38.394340 4235 ts_tablet_manager.cc:1397] T 801601d4b0a04720b6aaa927e16971ac P 830588ce3f974ac193a24845e0f79841: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 06:04:38.394708 4235 raft_consensus.cc:348] T 801601d4b0a04720b6aaa927e16971ac P 830588ce3f974ac193a24845e0f79841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } } peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } }
I20260808 06:04:38.394798 4235 raft_consensus.cc:374] T 801601d4b0a04720b6aaa927e16971ac P 830588ce3f974ac193a24845e0f79841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:38.394829 4235 raft_consensus.cc:729] T 801601d4b0a04720b6aaa927e16971ac P 830588ce3f974ac193a24845e0f79841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 830588ce3f974ac193a24845e0f79841, State: Initialized, Role: FOLLOWER
I20260808 06:04:38.394939 4235 consensus_queue.cc:260] T 801601d4b0a04720b6aaa927e16971ac P 830588ce3f974ac193a24845e0f79841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } } peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } }
I20260808 06:04:38.395071 4233 tablet_bootstrap.cc:654] T 801601d4b0a04720b6aaa927e16971ac P 7fd2f1f56a9143fcb30708ea6a309094: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:38.395176 4235 ts_tablet_manager.cc:1428] T 801601d4b0a04720b6aaa927e16971ac P 830588ce3f974ac193a24845e0f79841: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 06:04:38.395699 4272 ts_tablet_manager.cc:1905] T 64c4b94cfc814833938c108f5538ad1b P 009c1b5a8ce94ce79613d0a5473b4eb0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:04:38.397728 4270 ts_tablet_manager.cc:1918] T 64c4b94cfc814833938c108f5538ad1b P 830588ce3f974ac193a24845e0f79841: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:04:38.397815 4270 log.cc:1196] T 64c4b94cfc814833938c108f5538ad1b P 830588ce3f974ac193a24845e0f79841: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-0/wal/wals/64c4b94cfc814833938c108f5538ad1b
I20260808 06:04:38.398119 4270 ts_tablet_manager.cc:1939] T 64c4b94cfc814833938c108f5538ad1b P 830588ce3f974ac193a24845e0f79841: Deleting consensus metadata
I20260808 06:04:38.399013 4272 ts_tablet_manager.cc:1918] T 64c4b94cfc814833938c108f5538ad1b P 009c1b5a8ce94ce79613d0a5473b4eb0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:04:38.399107 4272 log.cc:1196] T 64c4b94cfc814833938c108f5538ad1b P 009c1b5a8ce94ce79613d0a5473b4eb0: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-2/wal/wals/64c4b94cfc814833938c108f5538ad1b
I20260808 06:04:38.399204 4269 tablet_replica.cc:330] T 64c4b94cfc814833938c108f5538ad1b P 7fd2f1f56a9143fcb30708ea6a309094: stopping tablet replica
I20260808 06:04:38.399174 3396 catalog_manager.cc:4841] TS 830588ce3f974ac193a24845e0f79841 (127.1.151.129:39609): tablet 64c4b94cfc814833938c108f5538ad1b (table test-table [id=813fc757222e47b896ecf435ae7480e2]) successfully deleted
I20260808 06:04:38.399437 4272 ts_tablet_manager.cc:1939] T 64c4b94cfc814833938c108f5538ad1b P 009c1b5a8ce94ce79613d0a5473b4eb0: Deleting consensus metadata
I20260808 06:04:38.399493 4269 raft_consensus.cc:2229] T 64c4b94cfc814833938c108f5538ad1b P 7fd2f1f56a9143fcb30708ea6a309094 [term 1 LEADER]: Raft consensus shutting down.
I20260808 06:04:38.399863 4269 raft_consensus.cc:2258] T 64c4b94cfc814833938c108f5538ad1b P 7fd2f1f56a9143fcb30708ea6a309094 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:04:38.400408 4234 tablet_bootstrap.cc:492] T 801601d4b0a04720b6aaa927e16971ac P 009c1b5a8ce94ce79613d0a5473b4eb0: No bootstrap required, opened a new log
I20260808 06:04:38.400488 4234 ts_tablet_manager.cc:1397] T 801601d4b0a04720b6aaa927e16971ac P 009c1b5a8ce94ce79613d0a5473b4eb0: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260808 06:04:38.400944 4234 raft_consensus.cc:348] T 801601d4b0a04720b6aaa927e16971ac P 009c1b5a8ce94ce79613d0a5473b4eb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } } peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } }
I20260808 06:04:38.401055 4234 raft_consensus.cc:374] T 801601d4b0a04720b6aaa927e16971ac P 009c1b5a8ce94ce79613d0a5473b4eb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:38.401085 4234 raft_consensus.cc:729] T 801601d4b0a04720b6aaa927e16971ac P 009c1b5a8ce94ce79613d0a5473b4eb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 009c1b5a8ce94ce79613d0a5473b4eb0, State: Initialized, Role: FOLLOWER
I20260808 06:04:38.401217 4234 consensus_queue.cc:260] T 801601d4b0a04720b6aaa927e16971ac P 009c1b5a8ce94ce79613d0a5473b4eb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } } peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } }
I20260808 06:04:38.401368 4269 ts_tablet_manager.cc:1905] T 64c4b94cfc814833938c108f5538ad1b P 7fd2f1f56a9143fcb30708ea6a309094: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:04:38.401458 4234 ts_tablet_manager.cc:1428] T 801601d4b0a04720b6aaa927e16971ac P 009c1b5a8ce94ce79613d0a5473b4eb0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 06:04:38.402513 3396 catalog_manager.cc:4841] TS 009c1b5a8ce94ce79613d0a5473b4eb0 (127.1.151.131:43093): tablet 64c4b94cfc814833938c108f5538ad1b (table test-table [id=813fc757222e47b896ecf435ae7480e2]) successfully deleted
I20260808 06:04:38.403360 4233 tablet_bootstrap.cc:492] T 801601d4b0a04720b6aaa927e16971ac P 7fd2f1f56a9143fcb30708ea6a309094: No bootstrap required, opened a new log
I20260808 06:04:38.403431 4233 ts_tablet_manager.cc:1397] T 801601d4b0a04720b6aaa927e16971ac P 7fd2f1f56a9143fcb30708ea6a309094: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260808 06:04:38.403841 4233 raft_consensus.cc:348] T 801601d4b0a04720b6aaa927e16971ac P 7fd2f1f56a9143fcb30708ea6a309094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } } peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } }
I20260808 06:04:38.403947 4233 raft_consensus.cc:374] T 801601d4b0a04720b6aaa927e16971ac P 7fd2f1f56a9143fcb30708ea6a309094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:38.404019 4233 raft_consensus.cc:729] T 801601d4b0a04720b6aaa927e16971ac P 7fd2f1f56a9143fcb30708ea6a309094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd2f1f56a9143fcb30708ea6a309094, State: Initialized, Role: FOLLOWER
I20260808 06:04:38.404146 4233 consensus_queue.cc:260] T 801601d4b0a04720b6aaa927e16971ac P 7fd2f1f56a9143fcb30708ea6a309094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } } peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } }
I20260808 06:04:38.404458 4233 ts_tablet_manager.cc:1428] T 801601d4b0a04720b6aaa927e16971ac P 7fd2f1f56a9143fcb30708ea6a309094: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 06:04:38.403939 4269 ts_tablet_manager.cc:1918] T 64c4b94cfc814833938c108f5538ad1b P 7fd2f1f56a9143fcb30708ea6a309094: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:04:38.404976 4269 log.cc:1196] T 64c4b94cfc814833938c108f5538ad1b P 7fd2f1f56a9143fcb30708ea6a309094: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-1/wal/wals/64c4b94cfc814833938c108f5538ad1b
I20260808 06:04:38.405316 4269 ts_tablet_manager.cc:1939] T 64c4b94cfc814833938c108f5538ad1b P 7fd2f1f56a9143fcb30708ea6a309094: Deleting consensus metadata
I20260808 06:04:38.406404 3400 catalog_manager.cc:4841] TS 7fd2f1f56a9143fcb30708ea6a309094 (127.1.151.130:44875): tablet 64c4b94cfc814833938c108f5538ad1b (table test-table [id=813fc757222e47b896ecf435ae7480e2]) successfully deleted
I20260808 06:04:38.432439 4245 raft_consensus.cc:482] T 801601d4b0a04720b6aaa927e16971ac P 009c1b5a8ce94ce79613d0a5473b4eb0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:04:38.432590 4245 raft_consensus.cc:504] T 801601d4b0a04720b6aaa927e16971ac P 009c1b5a8ce94ce79613d0a5473b4eb0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } } peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } }
I20260808 06:04:38.433831 4245 leader_election.cc:288] T 801601d4b0a04720b6aaa927e16971ac P 009c1b5a8ce94ce79613d0a5473b4eb0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 830588ce3f974ac193a24845e0f79841 (127.1.151.129:39609), 7fd2f1f56a9143fcb30708ea6a309094 (127.1.151.130:44875)
I20260808 06:04:38.438679 4226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:38.434340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:40973 (local address 127.1.151.130:44875)
0808 06:04:38.434449 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:38.434454 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:38.434582 (+ 128us) server_negotiation.cc:407] Connection header received
0808 06:04:38.435043 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:38.435046 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:38.435095 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:38.435154 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:04:38.435396 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.435881 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.437617 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.438013 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.438117 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:38.438193 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:38.438490 (+ 297us) server_negotiation.cc:299] Negotiation successful
0808 06:04:38.438554 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 06:04:38.438701 4278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:38.434008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:44875 (local address 127.1.151.131:40973)
0808 06:04:38.434496 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:38.434508 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:38.434648 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:38.435174 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:38.435177 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:38.435192 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:04:38.435294 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:38.435300 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.435979 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.435983 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:38.437462 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:38.437474 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.438130 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.438134 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:38.438372 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:38.438400 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 06:04:38.438472 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 06:04:38.438544 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"mutex_wait_us":160,"thread_start_us":68,"threads_started":1}
I20260808 06:04:38.438899 4019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:38.434007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:39609 (local address 127.1.151.131:49123)
0808 06:04:38.434131 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:38.434177 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:38.434268 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:38.434878 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:38.434880 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:38.434896 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:04:38.434994 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:38.434999 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.436027 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.436030 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:38.437467 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:38.437474 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.438492 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.438496 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:38.438614 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:38.438635 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 06:04:38.438701 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 06:04:38.438771 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":122}
I20260808 06:04:38.439381 4230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:38.434480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:49123 (local address 127.1.151.129:39609)
0808 06:04:38.434579 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:38.434584 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:38.434598 (+ 14us) server_negotiation.cc:407] Connection header received
0808 06:04:38.434655 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:38.434658 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:38.434705 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:38.434773 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:04:38.435118 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.435929 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.437597 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.438270 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.438383 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:38.438433 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:38.439171 (+ 738us) server_negotiation.cc:299] Negotiation successful
0808 06:04:38.439260 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 06:04:38.439399 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "801601d4b0a04720b6aaa927e16971ac" candidate_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd2f1f56a9143fcb30708ea6a309094" is_pre_election: true
I20260808 06:04:38.439667 3891 raft_consensus.cc:2454] T 801601d4b0a04720b6aaa927e16971ac P 7fd2f1f56a9143fcb30708ea6a309094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 009c1b5a8ce94ce79613d0a5473b4eb0 in term 0.
I20260808 06:04:38.439823 3673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "801601d4b0a04720b6aaa927e16971ac" candidate_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "830588ce3f974ac193a24845e0f79841" is_pre_election: true
I20260808 06:04:38.439946 3673 raft_consensus.cc:2454] T 801601d4b0a04720b6aaa927e16971ac P 830588ce3f974ac193a24845e0f79841 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 009c1b5a8ce94ce79613d0a5473b4eb0 in term 0.
I20260808 06:04:38.440325 4011 leader_election.cc:302] T 801601d4b0a04720b6aaa927e16971ac P 009c1b5a8ce94ce79613d0a5473b4eb0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 009c1b5a8ce94ce79613d0a5473b4eb0, 7fd2f1f56a9143fcb30708ea6a309094; no voters:
I20260808 06:04:38.442255 4245 raft_consensus.cc:2781] T 801601d4b0a04720b6aaa927e16971ac P 009c1b5a8ce94ce79613d0a5473b4eb0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 06:04:38.442313 4245 raft_consensus.cc:482] T 801601d4b0a04720b6aaa927e16971ac P 009c1b5a8ce94ce79613d0a5473b4eb0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 06:04:38.442356 4245 raft_consensus.cc:3037] T 801601d4b0a04720b6aaa927e16971ac P 009c1b5a8ce94ce79613d0a5473b4eb0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:38.443420 4245 raft_consensus.cc:504] T 801601d4b0a04720b6aaa927e16971ac P 009c1b5a8ce94ce79613d0a5473b4eb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } } peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } }
I20260808 06:04:38.443799 4245 leader_election.cc:288] T 801601d4b0a04720b6aaa927e16971ac P 009c1b5a8ce94ce79613d0a5473b4eb0 [CANDIDATE]: Term 1 election: Requested vote from peers 830588ce3f974ac193a24845e0f79841 (127.1.151.129:39609), 7fd2f1f56a9143fcb30708ea6a309094 (127.1.151.130:44875)
I20260808 06:04:38.444172 3673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "801601d4b0a04720b6aaa927e16971ac" candidate_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "830588ce3f974ac193a24845e0f79841"
I20260808 06:04:38.444208 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "801601d4b0a04720b6aaa927e16971ac" candidate_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd2f1f56a9143fcb30708ea6a309094"
I20260808 06:04:38.444300 3673 raft_consensus.cc:3037] T 801601d4b0a04720b6aaa927e16971ac P 830588ce3f974ac193a24845e0f79841 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:38.444303 3891 raft_consensus.cc:3037] T 801601d4b0a04720b6aaa927e16971ac P 7fd2f1f56a9143fcb30708ea6a309094 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:38.445441 3891 raft_consensus.cc:2454] T 801601d4b0a04720b6aaa927e16971ac P 7fd2f1f56a9143fcb30708ea6a309094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 009c1b5a8ce94ce79613d0a5473b4eb0 in term 1.
I20260808 06:04:38.445441 3673 raft_consensus.cc:2454] T 801601d4b0a04720b6aaa927e16971ac P 830588ce3f974ac193a24845e0f79841 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 009c1b5a8ce94ce79613d0a5473b4eb0 in term 1.
I20260808 06:04:38.445811 4005 leader_election.cc:302] T 801601d4b0a04720b6aaa927e16971ac P 009c1b5a8ce94ce79613d0a5473b4eb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 009c1b5a8ce94ce79613d0a5473b4eb0, 830588ce3f974ac193a24845e0f79841; no voters:
I20260808 06:04:38.446237 4245 raft_consensus.cc:2781] T 801601d4b0a04720b6aaa927e16971ac P 009c1b5a8ce94ce79613d0a5473b4eb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:04:38.446449 4245 raft_consensus.cc:686] T 801601d4b0a04720b6aaa927e16971ac P 009c1b5a8ce94ce79613d0a5473b4eb0 [term 1 LEADER]: Becoming Leader. State: Replica: 009c1b5a8ce94ce79613d0a5473b4eb0, State: Running, Role: LEADER
I20260808 06:04:38.446707 4245 consensus_queue.cc:237] T 801601d4b0a04720b6aaa927e16971ac P 009c1b5a8ce94ce79613d0a5473b4eb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } } peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } }
W20260808 06:04:38.447748 4190 tablet.cc:2369] T 801601d4b0a04720b6aaa927e16971ac P 009c1b5a8ce94ce79613d0a5473b4eb0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 06:04:38.449677 3417 catalog_manager.cc:5495] T 801601d4b0a04720b6aaa927e16971ac P 009c1b5a8ce94ce79613d0a5473b4eb0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 009c1b5a8ce94ce79613d0a5473b4eb0 (127.1.151.131). New cstate: current_term: 1 leader_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } health_report { overall_health: HEALTHY } } }
W20260808 06:04:38.463500 3960 tablet.cc:2369] T 801601d4b0a04720b6aaa927e16971ac P 7fd2f1f56a9143fcb30708ea6a309094: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 06:04:38 dist-test-slave-zj8q krb5kdc[3351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169077, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
W20260808 06:04:38.501482 3740 tablet.cc:2369] T 801601d4b0a04720b6aaa927e16971ac P 830588ce3f974ac193a24845e0f79841: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 06:04:38.504877 4287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:38.495829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39871 (local address 127.0.0.1:40524)
0808 06:04:38.496109 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:38.496125 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:38.496226 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:38.496482 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:38.496484 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:38.496883 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:38.496987 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:38.496993 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.498060 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.498062 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:38.498663 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:38.498672 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.499256 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.499259 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:38.499359 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:38.499972 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:04:38.499984 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:38.501491 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:38.503351 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:38.503360 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:38.503365 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:38.503583 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:38.503898 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:38.503899 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:38.503901 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:38.503973 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:38.504405 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:38.504410 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:38.504509 (+ 99us) client_negotiation.cc:769] Sending connection context
0808 06:04:38.504590 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 06:04:38.504737 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":2,"thread_start_us":85,"threads_started":1}
I20260808 06:04:38.505193 3502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:38.495918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40524 (local address 127.1.151.190:39871)
0808 06:04:38.496066 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:38.496071 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:38.496200 (+ 129us) server_negotiation.cc:407] Connection header received
0808 06:04:38.496311 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:38.496315 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:38.496366 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:38.496467 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:38.497108 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.497929 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.498781 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.499113 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.499261 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:38.501648 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:38.501652 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:38.501655 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:38.501681 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:38.503244 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:38.503710 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:38.503714 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:38.503715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:38.503812 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:38.504056 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:38.504060 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:38.504062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:38.504308 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:38.504411 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:38.504679 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 06:04:38.504812 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":63}
I20260808 06:04:38.509193 3417 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:40524:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 06:04:38.509922 3417 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 06:04:38.514667 3647 tablet_service.cc:1467] Processing CreateTablet for tablet e29b35ea70d145cca7f69ac758961402 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5c0c63e6d35b47fd964d92af745fb1b0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 06:04:38.514760 3858 tablet_service.cc:1467] Processing CreateTablet for tablet e29b35ea70d145cca7f69ac758961402 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5c0c63e6d35b47fd964d92af745fb1b0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 06:04:38.514850 4078 tablet_service.cc:1467] Processing CreateTablet for tablet e29b35ea70d145cca7f69ac758961402 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5c0c63e6d35b47fd964d92af745fb1b0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 06:04:38.515092 3647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e29b35ea70d145cca7f69ac758961402. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:38.515113 3858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e29b35ea70d145cca7f69ac758961402. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:38.515218 4078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e29b35ea70d145cca7f69ac758961402. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:38.517820 4233 tablet_bootstrap.cc:492] T e29b35ea70d145cca7f69ac758961402 P 7fd2f1f56a9143fcb30708ea6a309094: Bootstrap starting.
I20260808 06:04:38.518245 4235 tablet_bootstrap.cc:492] T e29b35ea70d145cca7f69ac758961402 P 830588ce3f974ac193a24845e0f79841: Bootstrap starting.
I20260808 06:04:38.518754 4233 tablet_bootstrap.cc:654] T e29b35ea70d145cca7f69ac758961402 P 7fd2f1f56a9143fcb30708ea6a309094: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:38.519070 4235 tablet_bootstrap.cc:654] T e29b35ea70d145cca7f69ac758961402 P 830588ce3f974ac193a24845e0f79841: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:38.519816 4234 tablet_bootstrap.cc:492] T e29b35ea70d145cca7f69ac758961402 P 009c1b5a8ce94ce79613d0a5473b4eb0: Bootstrap starting.
I20260808 06:04:38.520174 4235 tablet_bootstrap.cc:492] T e29b35ea70d145cca7f69ac758961402 P 830588ce3f974ac193a24845e0f79841: No bootstrap required, opened a new log
I20260808 06:04:38.520255 4235 ts_tablet_manager.cc:1397] T e29b35ea70d145cca7f69ac758961402 P 830588ce3f974ac193a24845e0f79841: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 06:04:38.520655 4235 raft_consensus.cc:348] T e29b35ea70d145cca7f69ac758961402 P 830588ce3f974ac193a24845e0f79841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } } peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } }
I20260808 06:04:38.520749 4235 raft_consensus.cc:374] T e29b35ea70d145cca7f69ac758961402 P 830588ce3f974ac193a24845e0f79841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:38.520763 4234 tablet_bootstrap.cc:654] T e29b35ea70d145cca7f69ac758961402 P 009c1b5a8ce94ce79613d0a5473b4eb0: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:38.520785 4235 raft_consensus.cc:729] T e29b35ea70d145cca7f69ac758961402 P 830588ce3f974ac193a24845e0f79841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 830588ce3f974ac193a24845e0f79841, State: Initialized, Role: FOLLOWER
I20260808 06:04:38.520912 4235 consensus_queue.cc:260] T e29b35ea70d145cca7f69ac758961402 P 830588ce3f974ac193a24845e0f79841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } } peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } }
I20260808 06:04:38.521144 4235 ts_tablet_manager.cc:1428] T e29b35ea70d145cca7f69ac758961402 P 830588ce3f974ac193a24845e0f79841: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 06:04:38.521407 4242 tablet_replica.cc:439] T e29b35ea70d145cca7f69ac758961402 P 830588ce3f974ac193a24845e0f79841: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } } peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } } }
I20260808 06:04:38.521540 4242 tablet_replica.cc:442] T e29b35ea70d145cca7f69ac758961402 P 830588ce3f974ac193a24845e0f79841: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 06:04:38.522614 4233 tablet_bootstrap.cc:492] T e29b35ea70d145cca7f69ac758961402 P 7fd2f1f56a9143fcb30708ea6a309094: No bootstrap required, opened a new log
I20260808 06:04:38.522698 4233 ts_tablet_manager.cc:1397] T e29b35ea70d145cca7f69ac758961402 P 7fd2f1f56a9143fcb30708ea6a309094: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 06:04:38.523147 4233 raft_consensus.cc:348] T e29b35ea70d145cca7f69ac758961402 P 7fd2f1f56a9143fcb30708ea6a309094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } } peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } }
I20260808 06:04:38.523259 4233 raft_consensus.cc:374] T e29b35ea70d145cca7f69ac758961402 P 7fd2f1f56a9143fcb30708ea6a309094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:38.523299 4233 raft_consensus.cc:729] T e29b35ea70d145cca7f69ac758961402 P 7fd2f1f56a9143fcb30708ea6a309094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd2f1f56a9143fcb30708ea6a309094, State: Initialized, Role: FOLLOWER
I20260808 06:04:38.523406 4233 consensus_queue.cc:260] T e29b35ea70d145cca7f69ac758961402 P 7fd2f1f56a9143fcb30708ea6a309094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } } peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } }
I20260808 06:04:38.523545 4234 tablet_bootstrap.cc:492] T e29b35ea70d145cca7f69ac758961402 P 009c1b5a8ce94ce79613d0a5473b4eb0: No bootstrap required, opened a new log
I20260808 06:04:38.523622 4234 ts_tablet_manager.cc:1397] T e29b35ea70d145cca7f69ac758961402 P 009c1b5a8ce94ce79613d0a5473b4eb0: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 06:04:38.523818 4243 tablet_replica.cc:439] T e29b35ea70d145cca7f69ac758961402 P 7fd2f1f56a9143fcb30708ea6a309094: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } } peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } } }
I20260808 06:04:38.523937 4243 tablet_replica.cc:442] T e29b35ea70d145cca7f69ac758961402 P 7fd2f1f56a9143fcb30708ea6a309094: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 06:04:38.524026 4234 raft_consensus.cc:348] T e29b35ea70d145cca7f69ac758961402 P 009c1b5a8ce94ce79613d0a5473b4eb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } } peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } }
I20260808 06:04:38.524158 4234 raft_consensus.cc:374] T e29b35ea70d145cca7f69ac758961402 P 009c1b5a8ce94ce79613d0a5473b4eb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:38.524190 4234 raft_consensus.cc:729] T e29b35ea70d145cca7f69ac758961402 P 009c1b5a8ce94ce79613d0a5473b4eb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 009c1b5a8ce94ce79613d0a5473b4eb0, State: Initialized, Role: FOLLOWER
I20260808 06:04:38.524302 4234 consensus_queue.cc:260] T e29b35ea70d145cca7f69ac758961402 P 009c1b5a8ce94ce79613d0a5473b4eb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } } peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } }
I20260808 06:04:38.524569 4234 ts_tablet_manager.cc:1428] T e29b35ea70d145cca7f69ac758961402 P 009c1b5a8ce94ce79613d0a5473b4eb0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 06:04:38.524526 4245 tablet_replica.cc:439] T e29b35ea70d145cca7f69ac758961402 P 009c1b5a8ce94ce79613d0a5473b4eb0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } } peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } } }
I20260808 06:04:38.524690 4245 tablet_replica.cc:442] T e29b35ea70d145cca7f69ac758961402 P 009c1b5a8ce94ce79613d0a5473b4eb0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 06:04:38.524940 4233 ts_tablet_manager.cc:1428] T e29b35ea70d145cca7f69ac758961402 P 7fd2f1f56a9143fcb30708ea6a309094: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 06:04:38.578094 4242 raft_consensus.cc:482] T e29b35ea70d145cca7f69ac758961402 P 830588ce3f974ac193a24845e0f79841 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:04:38.578262 4242 raft_consensus.cc:504] T e29b35ea70d145cca7f69ac758961402 P 830588ce3f974ac193a24845e0f79841 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } } peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } }
I20260808 06:04:38.579358 4242 leader_election.cc:288] T e29b35ea70d145cca7f69ac758961402 P 830588ce3f974ac193a24845e0f79841 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7fd2f1f56a9143fcb30708ea6a309094 (127.1.151.130:44875), 009c1b5a8ce94ce79613d0a5473b4eb0 (127.1.151.131:43093)
I20260808 06:04:38.583235 4226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:38.579663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:51375 (local address 127.1.151.130:44875)
0808 06:04:38.579788 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:38.579793 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:38.580035 (+ 242us) server_negotiation.cc:407] Connection header received
0808 06:04:38.580208 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:38.580212 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:38.580265 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:38.580328 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:04:38.580653 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.581173 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.582286 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.582685 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.582793 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:38.582847 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:38.583030 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 06:04:38.583113 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 06:04:38.583336 4225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:38.579759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:42591 (local address 127.1.151.131:43093)
0808 06:04:38.579912 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:38.579917 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:38.580092 (+ 175us) server_negotiation.cc:407] Connection header received
0808 06:04:38.580223 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:38.580225 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:38.580264 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:38.580325 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:04:38.580675 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.581167 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.582301 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.582706 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.582813 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:38.582864 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:38.583158 (+ 294us) server_negotiation.cc:299] Negotiation successful
0808 06:04:38.583229 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 06:04:38.583242 4292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:38.579554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:44875 (local address 127.1.151.129:51375)
0808 06:04:38.579934 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:38.579949 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:38.580061 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:38.580353 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:38.580356 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:38.580397 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:04:38.580543 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:38.580549 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.581277 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.581279 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:38.582140 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:38.582172 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.582798 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.582802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:38.582897 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:38.582921 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 06:04:38.582998 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 06:04:38.583064 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":26,"spinlock_wait_cycles":30848,"thread_start_us":140,"threads_started":1}
I20260808 06:04:38.583663 4294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:38.579598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:43093 (local address 127.1.151.129:42591)
0808 06:04:38.580018 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:38.580031 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:38.580123 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:38.580356 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:38.580359 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:38.580399 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:04:38.580556 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:38.580562 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.581274 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.581278 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:38.582189 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:38.582199 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.582916 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.582919 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:38.583014 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:38.583033 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 06:04:38.583455 (+ 422us) client_negotiation.cc:240] Negotiation successful
0808 06:04:38.583533 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":98,"spinlock_wait_cycles":4736,"thread_start_us":127,"threads_started":1}
I20260808 06:04:38.584017 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e29b35ea70d145cca7f69ac758961402" candidate_uuid: "830588ce3f974ac193a24845e0f79841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd2f1f56a9143fcb30708ea6a309094" is_pre_election: true
I20260808 06:04:38.584127 4112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e29b35ea70d145cca7f69ac758961402" candidate_uuid: "830588ce3f974ac193a24845e0f79841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" is_pre_election: true
I20260808 06:04:38.584182 3891 raft_consensus.cc:2454] T e29b35ea70d145cca7f69ac758961402 P 7fd2f1f56a9143fcb30708ea6a309094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 830588ce3f974ac193a24845e0f79841 in term 0.
I20260808 06:04:38.584266 4112 raft_consensus.cc:2454] T e29b35ea70d145cca7f69ac758961402 P 009c1b5a8ce94ce79613d0a5473b4eb0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 830588ce3f974ac193a24845e0f79841 in term 0.
I20260808 06:04:38.584585 3570 leader_election.cc:302] T e29b35ea70d145cca7f69ac758961402 P 830588ce3f974ac193a24845e0f79841 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd2f1f56a9143fcb30708ea6a309094, 830588ce3f974ac193a24845e0f79841; no voters:
I20260808 06:04:38.584846 4242 raft_consensus.cc:2781] T e29b35ea70d145cca7f69ac758961402 P 830588ce3f974ac193a24845e0f79841 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 06:04:38.584905 4242 raft_consensus.cc:482] T e29b35ea70d145cca7f69ac758961402 P 830588ce3f974ac193a24845e0f79841 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 06:04:38.584932 4242 raft_consensus.cc:3037] T e29b35ea70d145cca7f69ac758961402 P 830588ce3f974ac193a24845e0f79841 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:38.585935 4242 raft_consensus.cc:504] T e29b35ea70d145cca7f69ac758961402 P 830588ce3f974ac193a24845e0f79841 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } } peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } }
I20260808 06:04:38.586285 4242 leader_election.cc:288] T e29b35ea70d145cca7f69ac758961402 P 830588ce3f974ac193a24845e0f79841 [CANDIDATE]: Term 1 election: Requested vote from peers 7fd2f1f56a9143fcb30708ea6a309094 (127.1.151.130:44875), 009c1b5a8ce94ce79613d0a5473b4eb0 (127.1.151.131:43093)
I20260808 06:04:38.586596 4112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e29b35ea70d145cca7f69ac758961402" candidate_uuid: "830588ce3f974ac193a24845e0f79841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0"
I20260808 06:04:38.586616 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e29b35ea70d145cca7f69ac758961402" candidate_uuid: "830588ce3f974ac193a24845e0f79841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd2f1f56a9143fcb30708ea6a309094"
I20260808 06:04:38.586709 4112 raft_consensus.cc:3037] T e29b35ea70d145cca7f69ac758961402 P 009c1b5a8ce94ce79613d0a5473b4eb0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:38.586730 3891 raft_consensus.cc:3037] T e29b35ea70d145cca7f69ac758961402 P 7fd2f1f56a9143fcb30708ea6a309094 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:38.587594 3891 raft_consensus.cc:2454] T e29b35ea70d145cca7f69ac758961402 P 7fd2f1f56a9143fcb30708ea6a309094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 830588ce3f974ac193a24845e0f79841 in term 1.
I20260808 06:04:38.587579 4112 raft_consensus.cc:2454] T e29b35ea70d145cca7f69ac758961402 P 009c1b5a8ce94ce79613d0a5473b4eb0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 830588ce3f974ac193a24845e0f79841 in term 1.
I20260808 06:04:38.587857 3570 leader_election.cc:302] T e29b35ea70d145cca7f69ac758961402 P 830588ce3f974ac193a24845e0f79841 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 009c1b5a8ce94ce79613d0a5473b4eb0, 7fd2f1f56a9143fcb30708ea6a309094, 830588ce3f974ac193a24845e0f79841; no voters:
I20260808 06:04:38.588019 4242 raft_consensus.cc:2781] T e29b35ea70d145cca7f69ac758961402 P 830588ce3f974ac193a24845e0f79841 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:04:38.588222 4242 raft_consensus.cc:686] T e29b35ea70d145cca7f69ac758961402 P 830588ce3f974ac193a24845e0f79841 [term 1 LEADER]: Becoming Leader. State: Replica: 830588ce3f974ac193a24845e0f79841, State: Running, Role: LEADER
I20260808 06:04:38.588454 4242 consensus_queue.cc:237] T e29b35ea70d145cca7f69ac758961402 P 830588ce3f974ac193a24845e0f79841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } } peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } }
I20260808 06:04:38.589583 4298 tablet_replica.cc:439] T e29b35ea70d145cca7f69ac758961402 P 830588ce3f974ac193a24845e0f79841: TxnStatusTablet state changed. Reason: New leader 830588ce3f974ac193a24845e0f79841. Latest consensus state: current_term: 1 leader_uuid: "830588ce3f974ac193a24845e0f79841" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } } peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } } }
I20260808 06:04:38.589696 4298 tablet_replica.cc:442] T e29b35ea70d145cca7f69ac758961402 P 830588ce3f974ac193a24845e0f79841: This TxnStatusTablet replica's current role is: LEADER
I20260808 06:04:38.589967 4301 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 06:04:38.590126 4301 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 06:04:38.590745 3419 catalog_manager.cc:5495] T e29b35ea70d145cca7f69ac758961402 P 830588ce3f974ac193a24845e0f79841 reported cstate change: term changed from 0 to 1, leader changed from <none> to 830588ce3f974ac193a24845e0f79841 (127.1.151.129). New cstate: current_term: 1 leader_uuid: "830588ce3f974ac193a24845e0f79841" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } health_report { overall_health: UNKNOWN } } }
I20260808 06:04:38.607944 4230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:38.604233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45028 (local address 127.1.151.129:39609)
0808 06:04:38.604544 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:38.604547 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:38.604556 (+ 9us) server_negotiation.cc:407] Connection header received
0808 06:04:38.604605 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:38.604608 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:38.604653 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:38.604723 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:04:38.604999 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.605504 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.606718 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.606934 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.607005 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:38.607287 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:04:38.607645 (+ 358us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:04:38.607725 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:38.607803 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 06:04:38.607847 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243}
I20260808 06:04:38.608006 4287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:38.604159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:39609 (local address 127.0.0.1:45028)
0808 06:04:38.604302 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:38.604318 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:38.604402 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:38.604773 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:38.604777 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:38.604787 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 06:04:38.604899 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:38.604907 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.605614 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.605618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:38.606620 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:38.606631 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.607021 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.607025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:38.607119 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:38.607172 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 06:04:38.607720 (+ 548us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 06:04:38.607730 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 06:04:38.607790 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 06:04:38.607871 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 06:04:38.610767 4112 raft_consensus.cc:1261] T e29b35ea70d145cca7f69ac758961402 P 009c1b5a8ce94ce79613d0a5473b4eb0 [term 1 FOLLOWER]: Refusing update from remote peer 830588ce3f974ac193a24845e0f79841: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:04:38.611042 3891 raft_consensus.cc:1261] T e29b35ea70d145cca7f69ac758961402 P 7fd2f1f56a9143fcb30708ea6a309094 [term 1 FOLLOWER]: Refusing update from remote peer 830588ce3f974ac193a24845e0f79841: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:04:38.611207 4242 consensus_queue.cc:1035] T e29b35ea70d145cca7f69ac758961402 P 830588ce3f974ac193a24845e0f79841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:04:38.611541 4298 consensus_queue.cc:1035] T e29b35ea70d145cca7f69ac758961402 P 830588ce3f974ac193a24845e0f79841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:04:38.612593 4245 tablet_replica.cc:439] T e29b35ea70d145cca7f69ac758961402 P 009c1b5a8ce94ce79613d0a5473b4eb0: TxnStatusTablet state changed. Reason: New leader 830588ce3f974ac193a24845e0f79841. Latest consensus state: current_term: 1 leader_uuid: "830588ce3f974ac193a24845e0f79841" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } } peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } } }
I20260808 06:04:38.612713 4245 tablet_replica.cc:442] T e29b35ea70d145cca7f69ac758961402 P 009c1b5a8ce94ce79613d0a5473b4eb0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 06:04:38.613386 4243 tablet_replica.cc:439] T e29b35ea70d145cca7f69ac758961402 P 7fd2f1f56a9143fcb30708ea6a309094: TxnStatusTablet state changed. Reason: New leader 830588ce3f974ac193a24845e0f79841. Latest consensus state: current_term: 1 leader_uuid: "830588ce3f974ac193a24845e0f79841" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } } peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } } }
I20260808 06:04:38.613498 4243 tablet_replica.cc:442] T e29b35ea70d145cca7f69ac758961402 P 7fd2f1f56a9143fcb30708ea6a309094: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 06:04:38.614382 4298 tablet_replica.cc:439] T e29b35ea70d145cca7f69ac758961402 P 830588ce3f974ac193a24845e0f79841: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "830588ce3f974ac193a24845e0f79841" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } } peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } } }
I20260808 06:04:38.614481 4298 tablet_replica.cc:442] T e29b35ea70d145cca7f69ac758961402 P 830588ce3f974ac193a24845e0f79841: This TxnStatusTablet replica's current role is: LEADER
I20260808 06:04:38.614598 4255 mvcc.cc:205] Tried to move back new op lower bound from 7316148545984958464 to 7316148545900527616. Current Snapshot: MvccSnapshot[applied={T|T < 7316148545984958464 or (T in {7316148545984958464})}]
I20260808 06:04:38.615084 4245 tablet_replica.cc:439] T e29b35ea70d145cca7f69ac758961402 P 009c1b5a8ce94ce79613d0a5473b4eb0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "830588ce3f974ac193a24845e0f79841" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } } peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } } }
I20260808 06:04:38.615183 4245 tablet_replica.cc:442] T e29b35ea70d145cca7f69ac758961402 P 009c1b5a8ce94ce79613d0a5473b4eb0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 06:04:38.616852 4242 tablet_replica.cc:439] T e29b35ea70d145cca7f69ac758961402 P 830588ce3f974ac193a24845e0f79841: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "830588ce3f974ac193a24845e0f79841" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } } peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } } }
I20260808 06:04:38.616952 4242 tablet_replica.cc:442] T e29b35ea70d145cca7f69ac758961402 P 830588ce3f974ac193a24845e0f79841: This TxnStatusTablet replica's current role is: LEADER
I20260808 06:04:38.619184 4243 tablet_replica.cc:439] T e29b35ea70d145cca7f69ac758961402 P 7fd2f1f56a9143fcb30708ea6a309094: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "830588ce3f974ac193a24845e0f79841" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 } } peers { permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 } } peers { permanent_uuid: "009c1b5a8ce94ce79613d0a5473b4eb0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43093 } } }
I20260808 06:04:38.619290 4243 tablet_replica.cc:442] T e29b35ea70d145cca7f69ac758961402 P 7fd2f1f56a9143fcb30708ea6a309094: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 06:04:38.619385 4253 mvcc.cc:205] Tried to move back new op lower bound from 7316148545984958464 to 7316148545900527616. Current Snapshot: MvccSnapshot[applied={T|T < 7316148545984958464 or (T in {7316148545984958464})}]
I20260808 06:04:38.624799 4225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:38.621266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43712 (local address 127.1.151.131:43093)
0808 06:04:38.621374 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:38.621378 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:38.621433 (+ 55us) server_negotiation.cc:407] Connection header received
0808 06:04:38.621577 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:38.621580 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:38.621626 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:38.621732 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:04:38.621932 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.622455 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.623545 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.623760 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.623827 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:38.624065 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:04:38.624434 (+ 369us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:04:38.624512 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:38.624592 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 06:04:38.624650 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 06:04:38.634872 4202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:38.631660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:39609 (local address 127.1.151.131:48945)
0808 06:04:38.631889 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:38.631901 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:38.631973 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:38.632185 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:38.632187 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:38.632192 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 06:04:38.632281 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:38.632285 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.633192 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.633194 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:38.633753 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:38.633760 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.634308 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.634310 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:38.634376 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:38.634416 (+ 40us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 06:04:38.634683 (+ 267us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 06:04:38.634687 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 06:04:38.634729 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 06:04:38.634783 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138}
I20260808 06:04:38.634915 4230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:38.631774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:48945 (local address 127.1.151.129:39609)
0808 06:04:38.631884 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:38.631888 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:38.631944 (+ 56us) server_negotiation.cc:407] Connection header received
0808 06:04:38.632052 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:38.632055 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:38.632100 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:38.632171 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:04:38.632362 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.633105 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.633850 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.634214 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.634305 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:38.634527 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:04:38.634614 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:04:38.634681 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:38.634746 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 06:04:38.634796 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 06:04:38.641613 3891 raft_consensus.cc:1261] T 801601d4b0a04720b6aaa927e16971ac P 7fd2f1f56a9143fcb30708ea6a309094 [term 1 FOLLOWER]: Refusing update from remote peer 009c1b5a8ce94ce79613d0a5473b4eb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:04:38.641708 3673 raft_consensus.cc:1261] T 801601d4b0a04720b6aaa927e16971ac P 830588ce3f974ac193a24845e0f79841 [term 1 FOLLOWER]: Refusing update from remote peer 009c1b5a8ce94ce79613d0a5473b4eb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:04:38.642053 4279 consensus_queue.cc:1035] T 801601d4b0a04720b6aaa927e16971ac P 009c1b5a8ce94ce79613d0a5473b4eb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd2f1f56a9143fcb30708ea6a309094" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:04:38.642292 4245 consensus_queue.cc:1035] T 801601d4b0a04720b6aaa927e16971ac P 009c1b5a8ce94ce79613d0a5473b4eb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "830588ce3f974ac193a24845e0f79841" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:04:38.664289 4325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:38.660514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:43093 (local address 127.1.151.129:54047)
0808 06:04:38.660806 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:38.660817 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:38.660916 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:38.661111 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:38.661114 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:38.661122 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 06:04:38.661243 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:38.661250 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.662388 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.662391 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:38.663020 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:38.663029 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.663677 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.663679 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:38.663751 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:38.663793 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 06:04:38.664064 (+ 271us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 06:04:38.664067 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 06:04:38.664105 (+ 38us) client_negotiation.cc:240] Negotiation successful
0808 06:04:38.664165 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":104,"threads_started":1}
I20260808 06:04:38.664302 4225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:38.660581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:54047 (local address 127.1.151.131:43093)
0808 06:04:38.660690 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:38.660695 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:38.660883 (+ 188us) server_negotiation.cc:407] Connection header received
0808 06:04:38.660993 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:38.660996 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:38.661038 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:38.661096 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:04:38.661365 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.662145 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.663130 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.663522 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.663649 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:38.663914 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:04:38.663990 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:04:38.664064 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:38.664121 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 06:04:38.664186 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 06:04:38.691427 3419 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:04:38.691615 3419 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:04:38.693827 3419 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 00147b47a31b4ab5a2080efc0f364038: Sending DeleteTablet for 3 replicas of tablet 801601d4b0a04720b6aaa927e16971ac
I20260808 06:04:38.694432 4078 tablet_service.cc:1514] Processing DeleteTablet for tablet 801601d4b0a04720b6aaa927e16971ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:04:38 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:43704
I20260808 06:04:38.694613 4272 tablet_replica.cc:330] T 801601d4b0a04720b6aaa927e16971ac P 009c1b5a8ce94ce79613d0a5473b4eb0: stopping tablet replica
I20260808 06:04:38.694581 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet 801601d4b0a04720b6aaa927e16971ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:04:38 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:37740
I20260808 06:04:38.694727 4272 raft_consensus.cc:2229] T 801601d4b0a04720b6aaa927e16971ac P 009c1b5a8ce94ce79613d0a5473b4eb0 [term 1 LEADER]: Raft consensus shutting down.
I20260808 06:04:38.694780 4269 tablet_replica.cc:330] T 801601d4b0a04720b6aaa927e16971ac P 7fd2f1f56a9143fcb30708ea6a309094: stopping tablet replica
I20260808 06:04:38.694893 4269 raft_consensus.cc:2229] T 801601d4b0a04720b6aaa927e16971ac P 7fd2f1f56a9143fcb30708ea6a309094 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:04:38.694988 4272 raft_consensus.cc:2258] T 801601d4b0a04720b6aaa927e16971ac P 009c1b5a8ce94ce79613d0a5473b4eb0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:04:38.695003 4269 raft_consensus.cc:2258] T 801601d4b0a04720b6aaa927e16971ac P 7fd2f1f56a9143fcb30708ea6a309094 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:04:38.695456 3647 tablet_service.cc:1514] Processing DeleteTablet for tablet 801601d4b0a04720b6aaa927e16971ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:04:38 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:45022
I20260808 06:04:38.695622 4270 tablet_replica.cc:330] T 801601d4b0a04720b6aaa927e16971ac P 830588ce3f974ac193a24845e0f79841: stopping tablet replica
I20260808 06:04:38.695732 4270 raft_consensus.cc:2229] T 801601d4b0a04720b6aaa927e16971ac P 830588ce3f974ac193a24845e0f79841 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:04:38.695837 4270 raft_consensus.cc:2258] T 801601d4b0a04720b6aaa927e16971ac P 830588ce3f974ac193a24845e0f79841 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:04:38.696076 4269 ts_tablet_manager.cc:1905] T 801601d4b0a04720b6aaa927e16971ac P 7fd2f1f56a9143fcb30708ea6a309094: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:04:38.696437 4270 ts_tablet_manager.cc:1905] T 801601d4b0a04720b6aaa927e16971ac P 830588ce3f974ac193a24845e0f79841: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:04:38.698791 4272 ts_tablet_manager.cc:1905] T 801601d4b0a04720b6aaa927e16971ac P 009c1b5a8ce94ce79613d0a5473b4eb0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:04:38.699216 4270 ts_tablet_manager.cc:1918] T 801601d4b0a04720b6aaa927e16971ac P 830588ce3f974ac193a24845e0f79841: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 06:04:38.699286 4270 log.cc:1196] T 801601d4b0a04720b6aaa927e16971ac P 830588ce3f974ac193a24845e0f79841: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-0/wal/wals/801601d4b0a04720b6aaa927e16971ac
I20260808 06:04:38.699543 4270 ts_tablet_manager.cc:1939] T 801601d4b0a04720b6aaa927e16971ac P 830588ce3f974ac193a24845e0f79841: Deleting consensus metadata
I20260808 06:04:38.701351 4272 ts_tablet_manager.cc:1918] T 801601d4b0a04720b6aaa927e16971ac P 009c1b5a8ce94ce79613d0a5473b4eb0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 06:04:38.701442 4272 log.cc:1196] T 801601d4b0a04720b6aaa927e16971ac P 009c1b5a8ce94ce79613d0a5473b4eb0: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-2/wal/wals/801601d4b0a04720b6aaa927e16971ac
I20260808 06:04:38.701697 4272 ts_tablet_manager.cc:1939] T 801601d4b0a04720b6aaa927e16971ac P 009c1b5a8ce94ce79613d0a5473b4eb0: Deleting consensus metadata
I20260808 06:04:38.702368 3396 catalog_manager.cc:4841] TS 830588ce3f974ac193a24845e0f79841 (127.1.151.129:39609): tablet 801601d4b0a04720b6aaa927e16971ac (table test-table [id=b26da4514c934426bfb8fa55f281213f]) successfully deleted
I20260808 06:04:38.702543 3396 catalog_manager.cc:4841] TS 009c1b5a8ce94ce79613d0a5473b4eb0 (127.1.151.131:43093): tablet 801601d4b0a04720b6aaa927e16971ac (table test-table [id=b26da4514c934426bfb8fa55f281213f]) successfully deleted
I20260808 06:04:38.704593 4269 ts_tablet_manager.cc:1918] T 801601d4b0a04720b6aaa927e16971ac P 7fd2f1f56a9143fcb30708ea6a309094: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 06:04:38.704748 4269 log.cc:1196] T 801601d4b0a04720b6aaa927e16971ac P 7fd2f1f56a9143fcb30708ea6a309094: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786169070948234-1630-0/minicluster-data/ts-1/wal/wals/801601d4b0a04720b6aaa927e16971ac
I20260808 06:04:38.705051 4269 ts_tablet_manager.cc:1939] T 801601d4b0a04720b6aaa927e16971ac P 7fd2f1f56a9143fcb30708ea6a309094: Deleting consensus metadata
I20260808 06:04:38.705772 3400 catalog_manager.cc:4841] TS 7fd2f1f56a9143fcb30708ea6a309094 (127.1.151.130:44875): tablet 801601d4b0a04720b6aaa927e16971ac (table test-table [id=b26da4514c934426bfb8fa55f281213f]) successfully deleted
Aug 08 06:04:38 dist-test-slave-zj8q krb5kdc[3351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169078, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.151.129@KRBTEST.COM
I20260808 06:04:38.716559 4230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:38.703921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45030 (local address 127.1.151.129:39609)
0808 06:04:38.704029 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:38.704034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:38.704559 (+ 525us) server_negotiation.cc:407] Connection header received
0808 06:04:38.704753 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:38.704757 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:38.704801 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:38.704847 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:38.705824 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.706649 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.707949 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.708286 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.708356 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:38.712184 (+ 3828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:38.712194 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:38.712210 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:38.712237 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:38.714379 (+ 2142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:38.714922 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:38.714929 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:38.714931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:38.715039 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:38.715362 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:38.715366 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:38.715367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:38.715667 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:38.715857 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:38.716241 (+ 384us) server_negotiation.cc:299] Negotiation successful
0808 06:04:38.716370 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260808 06:04:38.717444 3727 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45030
I20260808 06:04:38.729112 3502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:38.719936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40540 (local address 127.1.151.190:39871)
0808 06:04:38.720095 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:38.720100 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:38.720252 (+ 152us) server_negotiation.cc:407] Connection header received
0808 06:04:38.720488 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:38.720491 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:38.720548 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:38.720650 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:38.721446 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.722299 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.723437 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:38.723787 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:38.723858 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:38.725457 (+ 1599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:38.725461 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:38.725464 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:38.725490 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:38.727280 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:38.727779 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:38.727783 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:38.727785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:38.727874 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:38.728169 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:38.728173 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:38.728175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:38.728373 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:38.728476 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:38.728839 (+ 363us) server_negotiation.cc:299] Negotiation successful
0808 06:04:38.728952 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260808 06:04:38.730111 3419 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40540
I20260808 06:04:38.731629 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 3543
I20260808 06:04:38.742962 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 3751
I20260808 06:04:38.759397 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 3979
I20260808 06:04:38.773298 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 3375
2026-08-08T06:04:38Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2771 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:04:38 dist-test-slave-zj8q krb5kdc[4339](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:04:38 dist-test-slave-zj8q krb5kdc[4339](info): set up 2 sockets
Aug 08 06:04:38 dist-test-slave-zj8q krb5kdc[4339](info): commencing operation
krb5kdc: starting...
W20260808 06:04:39.913046 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:04:39 dist-test-slave-zj8q krb5kdc[4339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169079, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:04:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:04:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.151.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
I20260808 06:04:40.030539 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:34837
--webserver_interface=127.1.151.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:39817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:34837
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:04:40.139530 4357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:40.139789 4357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:40.139829 4357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:40.143764 4357 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:04:40.143819 4357 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 06:04:40.143836 4357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:40.143851 4357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:40.143867 4357 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:04:40.143893 4357 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:04:40.149181 4357 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:39817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:34837
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:34837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:40.150398 4357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:40.151458 4357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:40.159832 4370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:40.159878 4365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:40.159914 4357 server_base.cc:1034] running on GCE node
W20260808 06:04:40.159865 4366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:40.160722 4357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:40.161763 4357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:40.162912 4357 hybrid_clock.cc:648] HybridClock initialized: now 1786169080162849 us; error 75 us; skew 500 ppm
Aug 08 06:04:40 dist-test-slave-zj8q krb5kdc[4339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169080, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:04:40.165587 4357 init.cc:377] Logged in from keytab as kudu/127.1.151.190@KRBTEST.COM (short username kudu)
I20260808 06:04:40.166455 4357 webserver.cc:460] Webserver started at http://127.1.151.190:43563/ using document root <none> and password file <none>
I20260808 06:04:40.166983 4357 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:40.167075 4357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:40.167289 4357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:40.169291 4357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "43452437015741b889520f1841311734"
format_stamp: "Formatted at 2026-08-08 06:04:40 on dist-test-slave-zj8q"
server_key: "4530fb879fe4381cd5d076711701c937"
server_key_iv: "03cc5fadb3140cb6e4d822ab629a5ccc"
server_key_version: "encryptionkey@0"
I20260808 06:04:40.169828 4357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "43452437015741b889520f1841311734"
format_stamp: "Formatted at 2026-08-08 06:04:40 on dist-test-slave-zj8q"
server_key: "4530fb879fe4381cd5d076711701c937"
server_key_iv: "03cc5fadb3140cb6e4d822ab629a5ccc"
server_key_version: "encryptionkey@0"
I20260808 06:04:40.173390 4357 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 06:04:40.175800 4378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:40.176988 4357 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260808 06:04:40.177130 4357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "43452437015741b889520f1841311734"
format_stamp: "Formatted at 2026-08-08 06:04:40 on dist-test-slave-zj8q"
server_key: "4530fb879fe4381cd5d076711701c937"
server_key_iv: "03cc5fadb3140cb6e4d822ab629a5ccc"
server_key_version: "encryptionkey@0"
I20260808 06:04:40.177227 4357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:40.202629 4357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:40.205754 4357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:40.205935 4357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:40.213373 4357 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:34837
I20260808 06:04:40.213389 4476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:34837 every 8 connection(s)
I20260808 06:04:40.214365 4357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:04:40.216133 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 4357
I20260808 06:04:40.216228 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:04:40.216481 1630 external_mini_cluster.cc:1428] Setting key 6f1ad1adb5ce1236fffa5c5b3d2be31d
I20260808 06:04:40.217235 4478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:40.223503 4478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43452437015741b889520f1841311734: Bootstrap starting.
Aug 08 06:04:40 dist-test-slave-zj8q krb5kdc[4339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169079, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:04:40.226133 4478 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43452437015741b889520f1841311734: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:40.227315 4478 log.cc:824] T 00000000000000000000000000000000 P 43452437015741b889520f1841311734: Log is configured to *not* fsync() on all Append() calls
I20260808 06:04:40.229357 4478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43452437015741b889520f1841311734: No bootstrap required, opened a new log
I20260808 06:04:40.230306 4481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:40.217772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40376 (local address 127.1.151.190:34837)
0808 06:04:40.218184 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:40.218209 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:40.218244 (+ 35us) server_negotiation.cc:407] Connection header received
0808 06:04:40.218857 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:40.218872 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:40.219183 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:40.219451 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:40.220364 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.221335 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.222525 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.223000 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.223137 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:40.225925 (+ 2788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:40.225930 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:40.225944 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:40.225971 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:40.227646 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:40.228066 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:40.228071 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:40.228076 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:40.228189 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:40.228460 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:40.228462 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:40.228463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:40.228849 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:40.228990 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:40.229275 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 06:04:40.229536 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":110,"threads_started":1}
I20260808 06:04:40.232779 4478 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43452437015741b889520f1841311734 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43452437015741b889520f1841311734" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34837 } }
I20260808 06:04:40.232990 4478 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43452437015741b889520f1841311734 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:40.233047 4478 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43452437015741b889520f1841311734 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43452437015741b889520f1841311734, State: Initialized, Role: FOLLOWER
I20260808 06:04:40.233453 4478 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43452437015741b889520f1841311734 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43452437015741b889520f1841311734" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34837 } }
I20260808 06:04:40.233588 4478 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43452437015741b889520f1841311734 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:04:40.233649 4478 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43452437015741b889520f1841311734 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:04:40.233744 4478 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43452437015741b889520f1841311734 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:40.234602 4478 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43452437015741b889520f1841311734 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43452437015741b889520f1841311734" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34837 } }
I20260808 06:04:40.234925 4478 leader_election.cc:302] T 00000000000000000000000000000000 P 43452437015741b889520f1841311734 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43452437015741b889520f1841311734; no voters:
I20260808 06:04:40.235204 4478 leader_election.cc:288] T 00000000000000000000000000000000 P 43452437015741b889520f1841311734 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:04:40.235322 4485 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43452437015741b889520f1841311734 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:04:40.235605 4485 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43452437015741b889520f1841311734 [term 1 LEADER]: Becoming Leader. State: Replica: 43452437015741b889520f1841311734, State: Running, Role: LEADER
I20260808 06:04:40.235900 4485 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43452437015741b889520f1841311734 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43452437015741b889520f1841311734" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34837 } }
I20260808 06:04:40.236414 4478 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43452437015741b889520f1841311734 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:04:40.237643 4488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43452437015741b889520f1841311734 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43452437015741b889520f1841311734. Latest consensus state: current_term: 1 leader_uuid: "43452437015741b889520f1841311734" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43452437015741b889520f1841311734" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34837 } } }
I20260808 06:04:40.237778 4488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43452437015741b889520f1841311734 [sys.catalog]: This master's current role is: LEADER
I20260808 06:04:40.237644 4486 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43452437015741b889520f1841311734 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43452437015741b889520f1841311734" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43452437015741b889520f1841311734" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34837 } } }
I20260808 06:04:40.238076 4486 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43452437015741b889520f1841311734 [sys.catalog]: This master's current role is: LEADER
I20260808 06:04:40.238279 4496 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:04:40.240933 4496 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:04:40.246404 4496 catalog_manager.cc:1347] Generated new cluster ID: 920e2b8fae014c1cb8101d33b353bcad
I20260808 06:04:40.246467 4496 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:04:40.256052 4496 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:04:40.256897 4496 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:04:40.269312 4496 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43452437015741b889520f1841311734: Generated new TSK 0
I20260808 06:04:40.269948 4496 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.151.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab.
Entry for principal kudu/127.1.151.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab.
I20260808 06:04:40.299379 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.129:0
--local_ip_for_outbound_sockets=127.1.151.129
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:34837
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:39817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:04:40.409547 4515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:40.409785 4515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:40.409849 4515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:40.413727 4515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:40.413779 4515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:40.413832 4515 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 06:04:40.413861 4515 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 06:04:40.413911 4515 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.129
I20260808 06:04:40.419009 4515 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:39817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.151.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.151.190:34837
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.129
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:40.419953 4515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:40.420897 4515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:40.429833 4523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:40.429850 4525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:40.429847 4522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:40.430289 4515 server_base.cc:1034] running on GCE node
I20260808 06:04:40.430630 4515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:40.431195 4515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:40.432350 4515 hybrid_clock.cc:648] HybridClock initialized: now 1786169080432333 us; error 30 us; skew 500 ppm
Aug 08 06:04:40 dist-test-slave-zj8q krb5kdc[4339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169080, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:04:40.435005 4515 init.cc:377] Logged in from keytab as kudu/127.1.151.129@KRBTEST.COM (short username kudu)
I20260808 06:04:40.435814 4515 webserver.cc:460] Webserver started at http://127.1.151.129:41137/ using document root <none> and password file <none>
I20260808 06:04:40.436316 4515 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:40.436375 4515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:40.436532 4515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:40.438614 4515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-0/data/instance:
uuid: "65412fe579b64b63b7626d30914841b5"
format_stamp: "Formatted at 2026-08-08 06:04:40 on dist-test-slave-zj8q"
server_key: "e00a91592a34226a0b5f9c5a1761046f"
server_key_iv: "90af1cdaba8e7299b924a5186de1e210"
server_key_version: "encryptionkey@0"
I20260808 06:04:40.439093 4515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance:
uuid: "65412fe579b64b63b7626d30914841b5"
format_stamp: "Formatted at 2026-08-08 06:04:40 on dist-test-slave-zj8q"
server_key: "e00a91592a34226a0b5f9c5a1761046f"
server_key_iv: "90af1cdaba8e7299b924a5186de1e210"
server_key_version: "encryptionkey@0"
I20260808 06:04:40.442461 4515 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 06:04:40.444612 4534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:40.445783 4515 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:04:40.445889 4515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-0/wal
uuid: "65412fe579b64b63b7626d30914841b5"
format_stamp: "Formatted at 2026-08-08 06:04:40 on dist-test-slave-zj8q"
server_key: "e00a91592a34226a0b5f9c5a1761046f"
server_key_iv: "90af1cdaba8e7299b924a5186de1e210"
server_key_version: "encryptionkey@0"
I20260808 06:04:40.445968 4515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:40.471500 4515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:40.474130 4515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:40.474293 4515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:40.475734 4515 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:04:40.475785 4515 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:40.475837 4515 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:04:40.475864 4515 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:40.485515 4515 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.129:34411
I20260808 06:04:40.485623 4716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.129:34411 every 8 connection(s)
Aug 08 06:04:40 dist-test-slave-zj8q krb5kdc[4339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169080, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:04:40.486462 4515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
I20260808 06:04:40.489832 4481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:40.476331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:45423 (local address 127.1.151.190:34837)
0808 06:04:40.476443 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:40.476448 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:40.477885 (+ 1437us) server_negotiation.cc:407] Connection header received
0808 06:04:40.479744 (+ 1859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:40.479748 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:40.479811 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:40.479898 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:40.481407 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.482262 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.483525 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.483872 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.483986 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:40.486608 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:40.486612 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:40.486615 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:40.486642 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:40.488046 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:40.488480 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:40.488483 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:40.488484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:40.488552 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:40.488865 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:40.488867 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:40.488868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:40.489033 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:40.489111 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:40.489570 (+ 459us) server_negotiation.cc:299] Negotiation successful
0808 06:04:40.489712 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 06:04:40.490607 4549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:40.477251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34837 (local address 127.1.151.129:45423)
0808 06:04:40.477723 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:40.477773 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:40.479516 (+ 1743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:40.480070 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:40.480079 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:40.480545 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:40.481224 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:40.481244 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.482393 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.482397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:40.483398 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:40.483408 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.484026 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.484031 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:40.484231 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:40.484804 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:04:40.484832 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:40.486447 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:40.488156 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:40.488160 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:40.488170 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:40.488405 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:40.488661 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:40.488663 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:40.488665 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:40.488797 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:40.489154 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:40.489160 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:40.489375 (+ 215us) client_negotiation.cc:769] Sending connection context
0808 06:04:40.489542 (+ 167us) client_negotiation.cc:240] Negotiation successful
0808 06:04:40.489776 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":111,"threads_started":1}
I20260808 06:04:40.491940 4717 heartbeater.cc:347] Connected to a master server at 127.1.151.190:34837
I20260808 06:04:40.492087 4717 heartbeater.cc:464] Registering TS with master...
I20260808 06:04:40.492647 4717 heartbeater.cc:511] Master 127.1.151.190:34837 requested a full tablet report, sending...
I20260808 06:04:40.494364 4408 ts_manager.cc:194] Registered new tserver with Master: 65412fe579b64b63b7626d30914841b5 (127.1.151.129:34411)
I20260808 06:04:40.495550 4408 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.129@KRBTEST.COM'} at 127.1.151.129:45423
I20260808 06:04:40.495990 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 4515
I20260808 06:04:40.496124 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance
I20260808 06:04:40.496447 1630 external_mini_cluster.cc:1428] Setting key ca20bb73001e08402175b6703d4b2e45
I20260808 06:04:40.500938 4725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:40.494139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34837 (local address 127.1.151.129:48139)
0808 06:04:40.494440 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:40.494453 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:40.494573 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:40.494803 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:40.494806 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:40.494979 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:40.495068 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:40.495073 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.495848 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.495851 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:40.496597 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:40.496607 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.497066 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.497070 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:40.497171 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:40.497581 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:04:40.497595 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:40.497960 (+ 365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:40.499528 (+ 1568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:40.499531 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:40.499533 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:40.499725 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:40.500022 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:40.500026 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:40.500029 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:40.500132 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:40.500496 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:40.500498 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:40.500624 (+ 126us) client_negotiation.cc:769] Sending connection context
0808 06:04:40.500709 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 06:04:40.500813 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":4,"thread_start_us":90,"threads_started":1}
I20260808 06:04:40.501245 4481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:40.494241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:48139 (local address 127.1.151.190:34837)
0808 06:04:40.494359 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:40.494363 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:40.494516 (+ 153us) server_negotiation.cc:407] Connection header received
0808 06:04:40.494675 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:40.494677 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:40.494715 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:40.494775 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:40.495197 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.495730 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.496701 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.496971 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.497060 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:40.498071 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:40.498074 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:40.498077 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:40.498100 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:40.499430 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:40.499818 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:40.499823 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:40.499824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:40.499914 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:40.500217 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:40.500220 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:40.500221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:40.500413 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:40.500498 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:40.500776 (+ 278us) server_negotiation.cc:299] Negotiation successful
0808 06:04:40.500883 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
WARNING: no policy specified for kudu/127.1.151.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab.
Entry for principal kudu/127.1.151.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab.
I20260808 06:04:40.523161 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.130:0
--local_ip_for_outbound_sockets=127.1.151.130
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:34837
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:39817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:04:40.633227 4731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:40.633446 4731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:40.633489 4731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:40.637418 4731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:40.637466 4731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:40.637502 4731 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 06:04:40.637518 4731 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 06:04:40.637571 4731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.130
I20260808 06:04:40.642649 4731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:39817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.151.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.151.190:34837
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4731
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.130
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:40.643505 4731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:40.644387 4731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:40.653475 4737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:40.653487 4743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:40.653489 4739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:40.653565 4731 server_base.cc:1034] running on GCE node
I20260808 06:04:40.654085 4731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:40.654721 4731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:40.655884 4731 hybrid_clock.cc:648] HybridClock initialized: now 1786169080655864 us; error 29 us; skew 500 ppm
Aug 08 06:04:40 dist-test-slave-zj8q krb5kdc[4339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169080, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:04:40.658460 4731 init.cc:377] Logged in from keytab as kudu/127.1.151.130@KRBTEST.COM (short username kudu)
I20260808 06:04:40.659296 4731 webserver.cc:460] Webserver started at http://127.1.151.130:46029/ using document root <none> and password file <none>
I20260808 06:04:40.659816 4731 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:40.659875 4731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:40.660068 4731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:40.662031 4731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-1/data/instance:
uuid: "62d68523dc894c159f9dc7abc1eff8d3"
format_stamp: "Formatted at 2026-08-08 06:04:40 on dist-test-slave-zj8q"
server_key: "838df0851ac6fbb118ce0f77b0a61515"
server_key_iv: "1723134ae3a9d6f56ab735c46efaadb7"
server_key_version: "encryptionkey@0"
I20260808 06:04:40.662621 4731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance:
uuid: "62d68523dc894c159f9dc7abc1eff8d3"
format_stamp: "Formatted at 2026-08-08 06:04:40 on dist-test-slave-zj8q"
server_key: "838df0851ac6fbb118ce0f77b0a61515"
server_key_iv: "1723134ae3a9d6f56ab735c46efaadb7"
server_key_version: "encryptionkey@0"
I20260808 06:04:40.666069 4731 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 06:04:40.668355 4752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:40.669492 4731 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:04:40.669610 4731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-1/wal
uuid: "62d68523dc894c159f9dc7abc1eff8d3"
format_stamp: "Formatted at 2026-08-08 06:04:40 on dist-test-slave-zj8q"
server_key: "838df0851ac6fbb118ce0f77b0a61515"
server_key_iv: "1723134ae3a9d6f56ab735c46efaadb7"
server_key_version: "encryptionkey@0"
I20260808 06:04:40.669709 4731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:40.685330 4731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:40.688203 4731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:40.688372 4731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:40.689764 4731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:04:40.689815 4731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:40.689867 4731 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:04:40.689893 4731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 06:04:40 dist-test-slave-zj8q krb5kdc[4339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169080, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:04:40.699599 4731 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.130:45285
I20260808 06:04:40.699710 4936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.130:45285 every 8 connection(s)
I20260808 06:04:40.700588 4731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
I20260808 06:04:40.703362 4481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:40.690428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:49885 (local address 127.1.151.190:34837)
0808 06:04:40.690596 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:40.690640 (+ 44us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:40.691847 (+ 1207us) server_negotiation.cc:407] Connection header received
0808 06:04:40.692860 (+ 1013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:40.692864 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:40.692926 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:40.693029 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:40.694464 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.695278 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.696877 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.697230 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.697344 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:40.700066 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:40.700070 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:40.700073 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:40.700102 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:40.701585 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:40.702062 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:40.702065 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:40.702066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:40.702168 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:40.702459 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:40.702461 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:40.702462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:40.702629 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:40.702729 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:40.703143 (+ 414us) server_negotiation.cc:299] Negotiation successful
0808 06:04:40.703243 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 06:04:40.704180 4763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:40.691252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34837 (local address 127.1.151.130:49885)
0808 06:04:40.691707 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:40.691756 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:40.692665 (+ 909us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:40.693207 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:40.693216 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:40.693649 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:40.694305 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:40.694325 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.695408 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.695411 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:40.696749 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:40.696759 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.697374 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.697376 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:40.697511 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:40.698214 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:04:40.698235 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:40.699907 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:40.701698 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:40.701702 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:40.701713 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:40.701975 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:40.702246 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:40.702248 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:40.702250 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:40.702389 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:40.702733 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:40.702738 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:40.702958 (+ 220us) client_negotiation.cc:769] Sending connection context
0808 06:04:40.703107 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 06:04:40.703352 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":109,"threads_started":1}
I20260808 06:04:40.705350 4938 heartbeater.cc:347] Connected to a master server at 127.1.151.190:34837
I20260808 06:04:40.705488 4938 heartbeater.cc:464] Registering TS with master...
I20260808 06:04:40.705996 4938 heartbeater.cc:511] Master 127.1.151.190:34837 requested a full tablet report, sending...
I20260808 06:04:40.707053 4409 ts_manager.cc:194] Registered new tserver with Master: 62d68523dc894c159f9dc7abc1eff8d3 (127.1.151.130:45285)
I20260808 06:04:40.707724 4409 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.130@KRBTEST.COM'} at 127.1.151.130:49885
I20260808 06:04:40.709125 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 4731
I20260808 06:04:40.709239 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance
I20260808 06:04:40.709504 1630 external_mini_cluster.cc:1428] Setting key a9a7daaf30ecd19b32e4255d9a8c3f3f
I20260808 06:04:40.713827 4481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:40.707362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:55131 (local address 127.1.151.190:34837)
0808 06:04:40.707462 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:40.707466 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:40.707672 (+ 206us) server_negotiation.cc:407] Connection header received
0808 06:04:40.707744 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:40.707747 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:40.707789 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:40.707854 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:40.708404 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.708924 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.709920 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.710144 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.710266 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:40.711183 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:40.711187 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:40.711189 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:40.711212 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:40.712350 (+ 1138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:40.712727 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:40.712730 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:40.712732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:40.712808 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:40.713049 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:40.713053 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:40.713054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:40.713260 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:40.713346 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:40.713591 (+ 245us) server_negotiation.cc:299] Negotiation successful
0808 06:04:40.713695 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 06:04:40.713845 4947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:40.707305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34837 (local address 127.1.151.130:55131)
0808 06:04:40.707574 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:40.707586 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:40.707672 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:40.707958 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:40.707961 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:40.708160 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:40.708276 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:40.708284 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.709066 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.709070 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:40.709812 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:40.709822 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.710269 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.710272 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:40.710364 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:40.710762 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:04:40.710775 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:40.711051 (+ 276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:40.712449 (+ 1398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:40.712452 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:40.712454 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:40.712632 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:40.712888 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:40.712891 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:40.712893 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:40.712971 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:40.713354 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:40.713356 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:40.713441 (+ 85us) client_negotiation.cc:769] Sending connection context
0808 06:04:40.713526 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 06:04:40.713667 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":113,"threads_started":1}
WARNING: no policy specified for kudu/127.1.151.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab.
Entry for principal kudu/127.1.151.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab.
I20260808 06:04:40.736554 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.131:0
--local_ip_for_outbound_sockets=127.1.151.131
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:34837
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:39817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:04:40.846956 4950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:40.847208 4950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:40.847254 4950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:40.851297 4950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:40.851349 4950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:40.851390 4950 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 06:04:40.851418 4950 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 06:04:40.851466 4950 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.131
I20260808 06:04:40.856705 4950 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:39817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.151.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.151.190:34837
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.131
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:40.857676 4950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:40.858680 4950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:40.868125 4962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:40.868134 4966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:40.868134 4963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:40.868323 4950 server_base.cc:1034] running on GCE node
I20260808 06:04:40.868727 4950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:40.869350 4950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:40.870509 4950 hybrid_clock.cc:648] HybridClock initialized: now 1786169080870489 us; error 31 us; skew 500 ppm
Aug 08 06:04:40 dist-test-slave-zj8q krb5kdc[4339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169080, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:04:40.873301 4950 init.cc:377] Logged in from keytab as kudu/127.1.151.131@KRBTEST.COM (short username kudu)
I20260808 06:04:40.874114 4950 webserver.cc:460] Webserver started at http://127.1.151.131:38899/ using document root <none> and password file <none>
I20260808 06:04:40.874716 4950 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:40.874778 4950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:40.874974 4950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:40.876920 4950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-2/data/instance:
uuid: "95ab12dd45364e63a84fbbc433f8ccf0"
format_stamp: "Formatted at 2026-08-08 06:04:40 on dist-test-slave-zj8q"
server_key: "c6a73cd339c0368b85223f2ff4f9eea2"
server_key_iv: "88eabc29a8970ec81c060f0b263d811b"
server_key_version: "encryptionkey@0"
I20260808 06:04:40.877462 4950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance:
uuid: "95ab12dd45364e63a84fbbc433f8ccf0"
format_stamp: "Formatted at 2026-08-08 06:04:40 on dist-test-slave-zj8q"
server_key: "c6a73cd339c0368b85223f2ff4f9eea2"
server_key_iv: "88eabc29a8970ec81c060f0b263d811b"
server_key_version: "encryptionkey@0"
I20260808 06:04:40.881070 4950 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 06:04:40.883443 4976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:40.884570 4950 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 06:04:40.884670 4950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-2/wal
uuid: "95ab12dd45364e63a84fbbc433f8ccf0"
format_stamp: "Formatted at 2026-08-08 06:04:40 on dist-test-slave-zj8q"
server_key: "c6a73cd339c0368b85223f2ff4f9eea2"
server_key_iv: "88eabc29a8970ec81c060f0b263d811b"
server_key_version: "encryptionkey@0"
I20260808 06:04:40.884768 4950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:40.902534 4950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:40.905264 4950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:40.905444 4950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:40.907021 4950 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:04:40.907070 4950 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:40.907109 4950 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:04:40.907131 4950 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 06:04:40 dist-test-slave-zj8q krb5kdc[4339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169080, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:04:40.916868 4950 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.131:34917
I20260808 06:04:40.917346 5168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.131:34917 every 8 connection(s)
I20260808 06:04:40.917896 4950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
I20260808 06:04:40.919791 4481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:40.907697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:51933 (local address 127.1.151.190:34837)
0808 06:04:40.907856 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:40.907861 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:40.908656 (+ 795us) server_negotiation.cc:407] Connection header received
0808 06:04:40.909561 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:40.909565 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:40.909630 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:40.909708 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:40.910983 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.911513 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.912684 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.912896 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.912981 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:40.916142 (+ 3161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:40.916146 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:40.916149 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:40.916174 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:40.917728 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:40.918244 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:40.918249 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:40.918250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:40.918352 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:40.918775 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:40.918778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:40.918780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:40.918959 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:40.919044 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:40.919514 (+ 470us) server_negotiation.cc:299] Negotiation successful
0808 06:04:40.919631 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 06:04:40.920464 4989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:40.908055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34837 (local address 127.1.151.131:51933)
0808 06:04:40.908495 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:40.908541 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:40.909357 (+ 816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:40.910003 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:40.910012 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:40.910384 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:40.910832 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:40.910846 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.911637 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.911640 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:40.912541 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:40.912553 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.913037 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.913041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:40.913226 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:40.913748 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:04:40.913767 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:40.915979 (+ 2212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:40.917834 (+ 1855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:40.917839 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:40.917849 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:40.918127 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:40.918468 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:40.918471 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:40.918473 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:40.918666 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:40.919060 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:40.919067 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:40.919286 (+ 219us) client_negotiation.cc:769] Sending connection context
0808 06:04:40.919431 (+ 145us) client_negotiation.cc:240] Negotiation successful
0808 06:04:40.919737 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":115,"threads_started":1}
I20260808 06:04:40.921517 5170 heartbeater.cc:347] Connected to a master server at 127.1.151.190:34837
I20260808 06:04:40.921653 5170 heartbeater.cc:464] Registering TS with master...
I20260808 06:04:40.922089 5170 heartbeater.cc:511] Master 127.1.151.190:34837 requested a full tablet report, sending...
I20260808 06:04:40.922901 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 4950
I20260808 06:04:40.923023 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance
I20260808 06:04:40.923090 4408 ts_manager.cc:194] Registered new tserver with Master: 95ab12dd45364e63a84fbbc433f8ccf0 (127.1.151.131:34917)
I20260808 06:04:40.923352 1630 external_mini_cluster.cc:1428] Setting key ec8d16f913ea1ca1af081505ded3c488
I20260808 06:04:40.923779 4408 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.131@KRBTEST.COM'} at 127.1.151.131:51933
I20260808 06:04:40.925398 1630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 06:04:40.929994 5179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:40.923485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34837 (local address 127.1.151.131:53965)
0808 06:04:40.923795 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:40.923809 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:40.923928 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:40.924224 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:40.924226 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:40.924394 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:40.924509 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:40.924516 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.925200 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.925203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:40.925812 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:40.925822 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.926243 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.926247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:40.926352 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:40.926728 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:04:40.926739 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:40.927022 (+ 283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:40.928700 (+ 1678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:40.928703 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:40.928704 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:40.928898 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:40.929165 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:40.929167 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:40.929168 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:40.929233 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:40.929586 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:40.929588 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:40.929679 (+ 91us) client_negotiation.cc:769] Sending connection context
0808 06:04:40.929753 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 06:04:40.929867 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":1,"thread_start_us":80,"threads_started":1}
I20260808 06:04:40.930119 4481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:40.923525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:53965 (local address 127.1.151.190:34837)
0808 06:04:40.923655 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:40.923659 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:40.923867 (+ 208us) server_negotiation.cc:407] Connection header received
0808 06:04:40.924027 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:40.924030 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:40.924073 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:40.924118 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:40.924609 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.925081 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.925917 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.926134 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.926241 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:40.927151 (+ 910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:40.927155 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:40.927157 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:40.927178 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:40.928560 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:40.928984 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:40.928988 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:40.928990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:40.929076 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:40.929306 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:40.929309 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:40.929310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:40.929506 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:40.929595 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:40.929869 (+ 274us) server_negotiation.cc:299] Negotiation successful
0808 06:04:40.929976 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
Aug 08 06:04:40 dist-test-slave-zj8q krb5kdc[4339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169080, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 06:04:40 dist-test-slave-zj8q krb5kdc[4339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169080, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:04:40.951519 4481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:40.944180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40390 (local address 127.1.151.190:34837)
0808 06:04:40.944321 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:40.944324 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:40.944337 (+ 13us) server_negotiation.cc:407] Connection header received
0808 06:04:40.944374 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:40.944376 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:40.944411 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:40.944480 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:40.945243 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.945761 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.946665 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.946869 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.946939 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:40.949028 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:40.949030 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:40.949033 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:40.949055 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:40.950232 (+ 1177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:40.950572 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:40.950575 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:40.950576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:40.950647 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:40.950827 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:40.950828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:40.950829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:40.950984 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:40.951076 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:40.951275 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 06:04:40.951367 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 06:04:40.954399 4408 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40390:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 06:04:40.957387 4408 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 06:04:40.970873 5193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:40.965396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:34917 (local address 127.0.0.1:50340)
0808 06:04:40.965752 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:40.965806 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:40.965927 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:40.967068 (+ 1141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:40.967075 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:40.967097 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:04:40.967217 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:40.967224 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.968316 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.968320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:40.969490 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:40.969500 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.970354 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.970357 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:40.970465 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:40.970499 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 06:04:40.970604 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 06:04:40.970718 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":128,"thread_start_us":70,"threads_started":1}
I20260808 06:04:40.970870 5195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:40.965563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50340 (local address 127.1.151.131:34917)
0808 06:04:40.966622 (+ 1059us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:40.966634 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:40.966655 (+ 21us) server_negotiation.cc:407] Connection header received
0808 06:04:40.966732 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:40.966737 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:40.966926 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:40.967071 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:04:40.967320 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.968218 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.969625 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.970249 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.970346 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:40.970496 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:40.970629 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 06:04:40.970721 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":953,"thread_start_us":116,"threads_started":1}
I20260808 06:04:40.971735 5196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:40.966115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:34411 (local address 127.0.0.1:60122)
0808 06:04:40.966401 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:40.966414 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:40.966500 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:40.967499 (+ 999us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:40.967502 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:40.967520 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:04:40.967629 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:40.967636 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.969008 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.969011 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:40.970291 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:40.970302 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.971348 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.971352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:40.971448 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:40.971472 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 06:04:40.971546 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 06:04:40.971609 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":1,"thread_start_us":122,"threads_started":1}
I20260808 06:04:40.971966 5194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:40.965447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:45285 (local address 127.0.0.1:50562)
0808 06:04:40.966274 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:40.966288 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:40.966374 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:40.967629 (+ 1255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:40.967632 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:40.967650 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:04:40.967746 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:40.967752 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.969208 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.969211 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:40.970355 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:40.970363 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.971613 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.971616 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:40.971704 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:40.971728 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 06:04:40.971780 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 06:04:40.971841 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":765,"mutex_wait_us":21,"thread_start_us":100,"threads_started":1}
I20260808 06:04:40.972645 5197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:40.966226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50562 (local address 127.1.151.130:45285)
0808 06:04:40.967241 (+ 1015us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:40.967251 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:40.967270 (+ 19us) server_negotiation.cc:407] Connection header received
0808 06:04:40.967331 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:40.967335 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:40.967487 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:40.967892 (+ 405us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:04:40.968030 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.969079 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.970469 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.971484 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.972249 (+ 765us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:40.972383 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:40.972431 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 06:04:40.972511 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":920,"thread_start_us":103,"threads_started":1}
I20260808 06:04:40.973536 5199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:40.966815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60122 (local address 127.1.151.129:34411)
0808 06:04:40.967077 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:40.967093 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:40.967116 (+ 23us) server_negotiation.cc:407] Connection header received
0808 06:04:40.967192 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:40.967197 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:40.967376 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:40.967525 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:04:40.967728 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.968882 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.970432 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:40.971232 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:40.971332 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:40.971461 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:40.971577 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 06:04:40.971679 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":88,"threads_started":1}
I20260808 06:04:40.974021 5065 tablet_service.cc:1467] Processing CreateTablet for tablet 19ab362f2b5047b68f894cc446a22414 (DEFAULT_TABLE table=test-table [id=4b1d132eb62443b78f2c0f7f5ecb3e04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:04:40.975106 5065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19ab362f2b5047b68f894cc446a22414. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:40.975212 4835 tablet_service.cc:1467] Processing CreateTablet for tablet 19ab362f2b5047b68f894cc446a22414 (DEFAULT_TABLE table=test-table [id=4b1d132eb62443b78f2c0f7f5ecb3e04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:04:40.976284 4835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19ab362f2b5047b68f894cc446a22414. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:40.976068 4608 tablet_service.cc:1467] Processing CreateTablet for tablet 19ab362f2b5047b68f894cc446a22414 (DEFAULT_TABLE table=test-table [id=4b1d132eb62443b78f2c0f7f5ecb3e04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:04:40.976982 4608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19ab362f2b5047b68f894cc446a22414. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:40.982292 5202 tablet_bootstrap.cc:492] T 19ab362f2b5047b68f894cc446a22414 P 95ab12dd45364e63a84fbbc433f8ccf0: Bootstrap starting.
I20260808 06:04:40.983577 5203 tablet_bootstrap.cc:492] T 19ab362f2b5047b68f894cc446a22414 P 62d68523dc894c159f9dc7abc1eff8d3: Bootstrap starting.
I20260808 06:04:40.984457 5202 tablet_bootstrap.cc:654] T 19ab362f2b5047b68f894cc446a22414 P 95ab12dd45364e63a84fbbc433f8ccf0: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:40.984586 5206 tablet_bootstrap.cc:492] T 19ab362f2b5047b68f894cc446a22414 P 65412fe579b64b63b7626d30914841b5: Bootstrap starting.
I20260808 06:04:40.985354 5202 log.cc:824] T 19ab362f2b5047b68f894cc446a22414 P 95ab12dd45364e63a84fbbc433f8ccf0: Log is configured to *not* fsync() on all Append() calls
I20260808 06:04:40.986227 5203 tablet_bootstrap.cc:654] T 19ab362f2b5047b68f894cc446a22414 P 62d68523dc894c159f9dc7abc1eff8d3: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:40.986564 5206 tablet_bootstrap.cc:654] T 19ab362f2b5047b68f894cc446a22414 P 65412fe579b64b63b7626d30914841b5: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:40.987015 5203 log.cc:824] T 19ab362f2b5047b68f894cc446a22414 P 62d68523dc894c159f9dc7abc1eff8d3: Log is configured to *not* fsync() on all Append() calls
I20260808 06:04:40.987218 5206 log.cc:824] T 19ab362f2b5047b68f894cc446a22414 P 65412fe579b64b63b7626d30914841b5: Log is configured to *not* fsync() on all Append() calls
I20260808 06:04:40.987709 5202 tablet_bootstrap.cc:492] T 19ab362f2b5047b68f894cc446a22414 P 95ab12dd45364e63a84fbbc433f8ccf0: No bootstrap required, opened a new log
I20260808 06:04:40.987914 5202 ts_tablet_manager.cc:1397] T 19ab362f2b5047b68f894cc446a22414 P 95ab12dd45364e63a84fbbc433f8ccf0: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 06:04:40.989014 5203 tablet_bootstrap.cc:492] T 19ab362f2b5047b68f894cc446a22414 P 62d68523dc894c159f9dc7abc1eff8d3: No bootstrap required, opened a new log
I20260808 06:04:40.989040 5206 tablet_bootstrap.cc:492] T 19ab362f2b5047b68f894cc446a22414 P 65412fe579b64b63b7626d30914841b5: No bootstrap required, opened a new log
I20260808 06:04:40.989203 5203 ts_tablet_manager.cc:1397] T 19ab362f2b5047b68f894cc446a22414 P 62d68523dc894c159f9dc7abc1eff8d3: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260808 06:04:40.989264 5206 ts_tablet_manager.cc:1397] T 19ab362f2b5047b68f894cc446a22414 P 65412fe579b64b63b7626d30914841b5: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 06:04:40.991129 5202 raft_consensus.cc:348] T 19ab362f2b5047b68f894cc446a22414 P 95ab12dd45364e63a84fbbc433f8ccf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } }
I20260808 06:04:40.991405 5202 raft_consensus.cc:374] T 19ab362f2b5047b68f894cc446a22414 P 95ab12dd45364e63a84fbbc433f8ccf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:40.991462 5202 raft_consensus.cc:729] T 19ab362f2b5047b68f894cc446a22414 P 95ab12dd45364e63a84fbbc433f8ccf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95ab12dd45364e63a84fbbc433f8ccf0, State: Initialized, Role: FOLLOWER
I20260808 06:04:40.991935 5202 consensus_queue.cc:260] T 19ab362f2b5047b68f894cc446a22414 P 95ab12dd45364e63a84fbbc433f8ccf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } }
I20260808 06:04:40.992553 5170 heartbeater.cc:503] Master 127.1.151.190:34837 was elected leader, sending a full tablet report...
I20260808 06:04:40.992975 5202 ts_tablet_manager.cc:1428] T 19ab362f2b5047b68f894cc446a22414 P 95ab12dd45364e63a84fbbc433f8ccf0: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 06:04:40.993091 5206 raft_consensus.cc:348] T 19ab362f2b5047b68f894cc446a22414 P 65412fe579b64b63b7626d30914841b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } }
I20260808 06:04:40.993069 5203 raft_consensus.cc:348] T 19ab362f2b5047b68f894cc446a22414 P 62d68523dc894c159f9dc7abc1eff8d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } }
I20260808 06:04:40.993306 5206 raft_consensus.cc:374] T 19ab362f2b5047b68f894cc446a22414 P 65412fe579b64b63b7626d30914841b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:40.993306 5203 raft_consensus.cc:374] T 19ab362f2b5047b68f894cc446a22414 P 62d68523dc894c159f9dc7abc1eff8d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:40.993352 5206 raft_consensus.cc:729] T 19ab362f2b5047b68f894cc446a22414 P 65412fe579b64b63b7626d30914841b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65412fe579b64b63b7626d30914841b5, State: Initialized, Role: FOLLOWER
I20260808 06:04:40.993368 5203 raft_consensus.cc:729] T 19ab362f2b5047b68f894cc446a22414 P 62d68523dc894c159f9dc7abc1eff8d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62d68523dc894c159f9dc7abc1eff8d3, State: Initialized, Role: FOLLOWER
I20260808 06:04:40.993839 5203 consensus_queue.cc:260] T 19ab362f2b5047b68f894cc446a22414 P 62d68523dc894c159f9dc7abc1eff8d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } }
I20260808 06:04:40.993860 5206 consensus_queue.cc:260] T 19ab362f2b5047b68f894cc446a22414 P 65412fe579b64b63b7626d30914841b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } }
I20260808 06:04:40.994786 5203 ts_tablet_manager.cc:1428] T 19ab362f2b5047b68f894cc446a22414 P 62d68523dc894c159f9dc7abc1eff8d3: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260808 06:04:40.994786 5206 ts_tablet_manager.cc:1428] T 19ab362f2b5047b68f894cc446a22414 P 65412fe579b64b63b7626d30914841b5: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 06:04:40.994848 4717 heartbeater.cc:503] Master 127.1.151.190:34837 was elected leader, sending a full tablet report...
I20260808 06:04:40.995110 4938 heartbeater.cc:503] Master 127.1.151.190:34837 was elected leader, sending a full tablet report...
I20260808 06:04:41.128456 5215 raft_consensus.cc:482] T 19ab362f2b5047b68f894cc446a22414 P 62d68523dc894c159f9dc7abc1eff8d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:04:41.128684 5215 raft_consensus.cc:504] T 19ab362f2b5047b68f894cc446a22414 P 62d68523dc894c159f9dc7abc1eff8d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } }
I20260808 06:04:41.129892 5215 leader_election.cc:288] T 19ab362f2b5047b68f894cc446a22414 P 62d68523dc894c159f9dc7abc1eff8d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95ab12dd45364e63a84fbbc433f8ccf0 (127.1.151.131:34917), 65412fe579b64b63b7626d30914841b5 (127.1.151.129:34411)
I20260808 06:04:41.133569 5195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:41.130293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:53899 (local address 127.1.151.131:34917)
0808 06:04:41.130431 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:41.130436 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:41.130454 (+ 18us) server_negotiation.cc:407] Connection header received
0808 06:04:41.130515 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:41.130518 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:41.130570 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:41.130654 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:04:41.130987 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.131550 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.132605 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.133047 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.133157 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:41.133221 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:41.133383 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 06:04:41.133448 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 06:04:41.133569 4763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:41.130148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:34917 (local address 127.1.151.130:53899)
0808 06:04:41.130309 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:41.130325 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:41.130423 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:41.130707 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:41.130711 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:41.130778 (+ 67us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:04:41.130876 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:41.130881 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.131658 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.131662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:41.132483 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:41.132492 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.133162 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.133166 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:41.133277 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:41.133295 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 06:04:41.133359 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 06:04:41.133431 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"spinlock_wait_cycles":20352}
I20260808 06:04:41.134032 5199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:41.130485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:55603 (local address 127.1.151.129:34411)
0808 06:04:41.130603 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:41.130608 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:41.130698 (+ 90us) server_negotiation.cc:407] Connection header received
0808 06:04:41.130813 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:41.130816 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:41.130858 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:41.130918 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:04:41.131182 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.131680 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.133118 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.133496 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.133599 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:41.133657 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:41.133858 (+ 201us) server_negotiation.cc:299] Negotiation successful
0808 06:04:41.133928 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 06:04:41.134042 5216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:41.130292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:34411 (local address 127.1.151.130:55603)
0808 06:04:41.130628 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:41.130640 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:41.130729 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:41.130938 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:41.130941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:41.130956 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:04:41.131076 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:41.131081 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.131802 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.131806 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:41.132987 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:41.132998 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.133601 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.133604 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:41.133713 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:41.133736 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 06:04:41.133834 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 06:04:41.133903 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":139,"threads_started":1}
I20260808 06:04:41.134207 5094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19ab362f2b5047b68f894cc446a22414" candidate_uuid: "62d68523dc894c159f9dc7abc1eff8d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" is_pre_election: true
I20260808 06:04:41.134509 5094 raft_consensus.cc:2454] T 19ab362f2b5047b68f894cc446a22414 P 95ab12dd45364e63a84fbbc433f8ccf0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62d68523dc894c159f9dc7abc1eff8d3 in term 0.
I20260808 06:04:41.134634 4644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19ab362f2b5047b68f894cc446a22414" candidate_uuid: "62d68523dc894c159f9dc7abc1eff8d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65412fe579b64b63b7626d30914841b5" is_pre_election: true
I20260808 06:04:41.134994 4753 leader_election.cc:302] T 19ab362f2b5047b68f894cc446a22414 P 62d68523dc894c159f9dc7abc1eff8d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62d68523dc894c159f9dc7abc1eff8d3, 95ab12dd45364e63a84fbbc433f8ccf0; no voters:
I20260808 06:04:41.135010 4644 raft_consensus.cc:2454] T 19ab362f2b5047b68f894cc446a22414 P 65412fe579b64b63b7626d30914841b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62d68523dc894c159f9dc7abc1eff8d3 in term 0.
I20260808 06:04:41.135252 5215 raft_consensus.cc:2781] T 19ab362f2b5047b68f894cc446a22414 P 62d68523dc894c159f9dc7abc1eff8d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 06:04:41.135349 5215 raft_consensus.cc:482] T 19ab362f2b5047b68f894cc446a22414 P 62d68523dc894c159f9dc7abc1eff8d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 06:04:41.135396 5215 raft_consensus.cc:3037] T 19ab362f2b5047b68f894cc446a22414 P 62d68523dc894c159f9dc7abc1eff8d3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:41.136484 5215 raft_consensus.cc:504] T 19ab362f2b5047b68f894cc446a22414 P 62d68523dc894c159f9dc7abc1eff8d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } }
I20260808 06:04:41.136809 5215 leader_election.cc:288] T 19ab362f2b5047b68f894cc446a22414 P 62d68523dc894c159f9dc7abc1eff8d3 [CANDIDATE]: Term 1 election: Requested vote from peers 95ab12dd45364e63a84fbbc433f8ccf0 (127.1.151.131:34917), 65412fe579b64b63b7626d30914841b5 (127.1.151.129:34411)
I20260808 06:04:41.137132 4644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19ab362f2b5047b68f894cc446a22414" candidate_uuid: "62d68523dc894c159f9dc7abc1eff8d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65412fe579b64b63b7626d30914841b5"
I20260808 06:04:41.137156 5094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19ab362f2b5047b68f894cc446a22414" candidate_uuid: "62d68523dc894c159f9dc7abc1eff8d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95ab12dd45364e63a84fbbc433f8ccf0"
I20260808 06:04:41.137259 4644 raft_consensus.cc:3037] T 19ab362f2b5047b68f894cc446a22414 P 65412fe579b64b63b7626d30914841b5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:41.137293 5094 raft_consensus.cc:3037] T 19ab362f2b5047b68f894cc446a22414 P 95ab12dd45364e63a84fbbc433f8ccf0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:41.138325 4644 raft_consensus.cc:2454] T 19ab362f2b5047b68f894cc446a22414 P 65412fe579b64b63b7626d30914841b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62d68523dc894c159f9dc7abc1eff8d3 in term 1.
I20260808 06:04:41.138409 5094 raft_consensus.cc:2454] T 19ab362f2b5047b68f894cc446a22414 P 95ab12dd45364e63a84fbbc433f8ccf0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62d68523dc894c159f9dc7abc1eff8d3 in term 1.
I20260808 06:04:41.138620 4754 leader_election.cc:302] T 19ab362f2b5047b68f894cc446a22414 P 62d68523dc894c159f9dc7abc1eff8d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62d68523dc894c159f9dc7abc1eff8d3, 65412fe579b64b63b7626d30914841b5; no voters:
I20260808 06:04:41.138768 5215 raft_consensus.cc:2781] T 19ab362f2b5047b68f894cc446a22414 P 62d68523dc894c159f9dc7abc1eff8d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:04:41.138962 5215 raft_consensus.cc:686] T 19ab362f2b5047b68f894cc446a22414 P 62d68523dc894c159f9dc7abc1eff8d3 [term 1 LEADER]: Becoming Leader. State: Replica: 62d68523dc894c159f9dc7abc1eff8d3, State: Running, Role: LEADER
I20260808 06:04:41.139252 5215 consensus_queue.cc:237] T 19ab362f2b5047b68f894cc446a22414 P 62d68523dc894c159f9dc7abc1eff8d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } }
I20260808 06:04:41.142304 4408 catalog_manager.cc:5495] T 19ab362f2b5047b68f894cc446a22414 P 62d68523dc894c159f9dc7abc1eff8d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 62d68523dc894c159f9dc7abc1eff8d3 (127.1.151.130). New cstate: current_term: 1 leader_uuid: "62d68523dc894c159f9dc7abc1eff8d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } health_report { overall_health: UNKNOWN } } }
W20260808 06:04:41.169076 5172 tablet.cc:2369] T 19ab362f2b5047b68f894cc446a22414 P 95ab12dd45364e63a84fbbc433f8ccf0: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 06:04:41 dist-test-slave-zj8q krb5kdc[4339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169080, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:04:41.188105 5225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:41.175677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34837 (local address 127.0.0.1:40406)
0808 06:04:41.175957 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:41.175970 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:41.176062 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:41.176272 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:41.176274 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:41.176643 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:41.176755 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:41.176761 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.177463 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.177465 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:41.178028 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:41.178035 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.178456 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.178459 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:41.178526 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:41.182930 (+ 4404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:04:41.182943 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:41.184425 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:41.186436 (+ 2011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:41.186445 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:41.186453 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:41.186748 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:41.187023 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:41.187027 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:41.187029 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:41.187133 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:41.187603 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:41.187609 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:41.187733 (+ 124us) client_negotiation.cc:769] Sending connection context
0808 06:04:41.187822 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 06:04:41.187939 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":113,"threads_started":1}
I20260808 06:04:41.188114 4481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:41.175743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40406 (local address 127.1.151.190:34837)
0808 06:04:41.175880 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:41.175884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:41.176036 (+ 152us) server_negotiation.cc:407] Connection header received
0808 06:04:41.176131 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:41.176134 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:41.176176 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:41.176241 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:41.176870 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.177382 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.178109 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.178353 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.178430 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:41.184618 (+ 6188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:41.184622 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:41.184625 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:41.184661 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:41.186327 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:41.186835 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:41.186838 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:41.186840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:41.186947 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:41.187214 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:41.187218 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:41.187220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:41.187448 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:41.187542 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:41.187890 (+ 348us) server_negotiation.cc:299] Negotiation successful
0808 06:04:41.187993 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 06:04:41.191802 4408 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:40406:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 06:04:41.192528 4408 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 06:04:41.197359 4835 tablet_service.cc:1467] Processing CreateTablet for tablet 150991c056e34c9d961e0ee0b872324d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=14978dec215746e38dc2e4265b7ede9e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 06:04:41.197311 4608 tablet_service.cc:1467] Processing CreateTablet for tablet 150991c056e34c9d961e0ee0b872324d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=14978dec215746e38dc2e4265b7ede9e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 06:04:41.197438 5065 tablet_service.cc:1467] Processing CreateTablet for tablet 150991c056e34c9d961e0ee0b872324d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=14978dec215746e38dc2e4265b7ede9e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 06:04:41.197960 4835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 150991c056e34c9d961e0ee0b872324d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:41.198001 5065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 150991c056e34c9d961e0ee0b872324d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:41.198001 4608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 150991c056e34c9d961e0ee0b872324d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:41.200721 5203 tablet_bootstrap.cc:492] T 150991c056e34c9d961e0ee0b872324d P 62d68523dc894c159f9dc7abc1eff8d3: Bootstrap starting.
I20260808 06:04:41.200997 5206 tablet_bootstrap.cc:492] T 150991c056e34c9d961e0ee0b872324d P 65412fe579b64b63b7626d30914841b5: Bootstrap starting.
W20260808 06:04:41.201694 4940 tablet.cc:2369] T 19ab362f2b5047b68f894cc446a22414 P 62d68523dc894c159f9dc7abc1eff8d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 06:04:41.201740 5203 tablet_bootstrap.cc:654] T 150991c056e34c9d961e0ee0b872324d P 62d68523dc894c159f9dc7abc1eff8d3: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:41.201930 5206 tablet_bootstrap.cc:654] T 150991c056e34c9d961e0ee0b872324d P 65412fe579b64b63b7626d30914841b5: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:41.202468 5202 tablet_bootstrap.cc:492] T 150991c056e34c9d961e0ee0b872324d P 95ab12dd45364e63a84fbbc433f8ccf0: Bootstrap starting.
I20260808 06:04:41.203001 5203 tablet_bootstrap.cc:492] T 150991c056e34c9d961e0ee0b872324d P 62d68523dc894c159f9dc7abc1eff8d3: No bootstrap required, opened a new log
I20260808 06:04:41.203083 5203 ts_tablet_manager.cc:1397] T 150991c056e34c9d961e0ee0b872324d P 62d68523dc894c159f9dc7abc1eff8d3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 06:04:41.203243 5202 tablet_bootstrap.cc:654] T 150991c056e34c9d961e0ee0b872324d P 95ab12dd45364e63a84fbbc433f8ccf0: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:41.203542 5203 raft_consensus.cc:348] T 150991c056e34c9d961e0ee0b872324d P 62d68523dc894c159f9dc7abc1eff8d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } }
I20260808 06:04:41.203680 5203 raft_consensus.cc:374] T 150991c056e34c9d961e0ee0b872324d P 62d68523dc894c159f9dc7abc1eff8d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:41.203732 5203 raft_consensus.cc:729] T 150991c056e34c9d961e0ee0b872324d P 62d68523dc894c159f9dc7abc1eff8d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62d68523dc894c159f9dc7abc1eff8d3, State: Initialized, Role: FOLLOWER
I20260808 06:04:41.203863 5203 consensus_queue.cc:260] T 150991c056e34c9d961e0ee0b872324d P 62d68523dc894c159f9dc7abc1eff8d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } }
I20260808 06:04:41.204144 5203 ts_tablet_manager.cc:1428] T 150991c056e34c9d961e0ee0b872324d P 62d68523dc894c159f9dc7abc1eff8d3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 06:04:41.204114 5215 tablet_replica.cc:439] T 150991c056e34c9d961e0ee0b872324d P 62d68523dc894c159f9dc7abc1eff8d3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } } }
I20260808 06:04:41.204246 5215 tablet_replica.cc:442] T 150991c056e34c9d961e0ee0b872324d P 62d68523dc894c159f9dc7abc1eff8d3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 06:04:41.204428 5202 tablet_bootstrap.cc:492] T 150991c056e34c9d961e0ee0b872324d P 95ab12dd45364e63a84fbbc433f8ccf0: No bootstrap required, opened a new log
I20260808 06:04:41.204506 5202 ts_tablet_manager.cc:1397] T 150991c056e34c9d961e0ee0b872324d P 95ab12dd45364e63a84fbbc433f8ccf0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 06:04:41.204900 5202 raft_consensus.cc:348] T 150991c056e34c9d961e0ee0b872324d P 95ab12dd45364e63a84fbbc433f8ccf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } }
I20260808 06:04:41.205011 5202 raft_consensus.cc:374] T 150991c056e34c9d961e0ee0b872324d P 95ab12dd45364e63a84fbbc433f8ccf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:41.205057 5202 raft_consensus.cc:729] T 150991c056e34c9d961e0ee0b872324d P 95ab12dd45364e63a84fbbc433f8ccf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95ab12dd45364e63a84fbbc433f8ccf0, State: Initialized, Role: FOLLOWER
I20260808 06:04:41.205201 5202 consensus_queue.cc:260] T 150991c056e34c9d961e0ee0b872324d P 95ab12dd45364e63a84fbbc433f8ccf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } }
I20260808 06:04:41.205422 5202 ts_tablet_manager.cc:1428] T 150991c056e34c9d961e0ee0b872324d P 95ab12dd45364e63a84fbbc433f8ccf0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 06:04:41.205647 5212 tablet_replica.cc:439] T 150991c056e34c9d961e0ee0b872324d P 95ab12dd45364e63a84fbbc433f8ccf0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } } }
I20260808 06:04:41.205781 5212 tablet_replica.cc:442] T 150991c056e34c9d961e0ee0b872324d P 95ab12dd45364e63a84fbbc433f8ccf0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 06:04:41.206806 5206 tablet_bootstrap.cc:492] T 150991c056e34c9d961e0ee0b872324d P 65412fe579b64b63b7626d30914841b5: No bootstrap required, opened a new log
I20260808 06:04:41.206874 5206 ts_tablet_manager.cc:1397] T 150991c056e34c9d961e0ee0b872324d P 65412fe579b64b63b7626d30914841b5: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260808 06:04:41.207311 5206 raft_consensus.cc:348] T 150991c056e34c9d961e0ee0b872324d P 65412fe579b64b63b7626d30914841b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } }
I20260808 06:04:41.207405 5206 raft_consensus.cc:374] T 150991c056e34c9d961e0ee0b872324d P 65412fe579b64b63b7626d30914841b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:41.207427 5206 raft_consensus.cc:729] T 150991c056e34c9d961e0ee0b872324d P 65412fe579b64b63b7626d30914841b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65412fe579b64b63b7626d30914841b5, State: Initialized, Role: FOLLOWER
I20260808 06:04:41.207558 5206 consensus_queue.cc:260] T 150991c056e34c9d961e0ee0b872324d P 65412fe579b64b63b7626d30914841b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } }
I20260808 06:04:41.207782 5206 ts_tablet_manager.cc:1428] T 150991c056e34c9d961e0ee0b872324d P 65412fe579b64b63b7626d30914841b5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 06:04:41.207978 5213 tablet_replica.cc:439] T 150991c056e34c9d961e0ee0b872324d P 65412fe579b64b63b7626d30914841b5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } } }
I20260808 06:04:41.208103 5213 tablet_replica.cc:442] T 150991c056e34c9d961e0ee0b872324d P 65412fe579b64b63b7626d30914841b5: This TxnStatusTablet replica's current role is: FOLLOWER
W20260808 06:04:41.237859 4718 tablet.cc:2369] T 150991c056e34c9d961e0ee0b872324d P 65412fe579b64b63b7626d30914841b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 06:04:41.259147 5215 raft_consensus.cc:482] T 150991c056e34c9d961e0ee0b872324d P 62d68523dc894c159f9dc7abc1eff8d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:04:41.259267 5215 raft_consensus.cc:504] T 150991c056e34c9d961e0ee0b872324d P 62d68523dc894c159f9dc7abc1eff8d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } }
I20260808 06:04:41.259671 5215 leader_election.cc:288] T 150991c056e34c9d961e0ee0b872324d P 62d68523dc894c159f9dc7abc1eff8d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 65412fe579b64b63b7626d30914841b5 (127.1.151.129:34411), 95ab12dd45364e63a84fbbc433f8ccf0 (127.1.151.131:34917)
I20260808 06:04:41.260098 5094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "150991c056e34c9d961e0ee0b872324d" candidate_uuid: "62d68523dc894c159f9dc7abc1eff8d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" is_pre_election: true
I20260808 06:04:41.260113 4644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "150991c056e34c9d961e0ee0b872324d" candidate_uuid: "62d68523dc894c159f9dc7abc1eff8d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65412fe579b64b63b7626d30914841b5" is_pre_election: true
I20260808 06:04:41.260239 4644 raft_consensus.cc:2454] T 150991c056e34c9d961e0ee0b872324d P 65412fe579b64b63b7626d30914841b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62d68523dc894c159f9dc7abc1eff8d3 in term 0.
I20260808 06:04:41.260250 5094 raft_consensus.cc:2454] T 150991c056e34c9d961e0ee0b872324d P 95ab12dd45364e63a84fbbc433f8ccf0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62d68523dc894c159f9dc7abc1eff8d3 in term 0.
I20260808 06:04:41.260532 4754 leader_election.cc:302] T 150991c056e34c9d961e0ee0b872324d P 62d68523dc894c159f9dc7abc1eff8d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62d68523dc894c159f9dc7abc1eff8d3, 65412fe579b64b63b7626d30914841b5; no voters:
I20260808 06:04:41.260726 5215 raft_consensus.cc:2781] T 150991c056e34c9d961e0ee0b872324d P 62d68523dc894c159f9dc7abc1eff8d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 06:04:41.260782 5215 raft_consensus.cc:482] T 150991c056e34c9d961e0ee0b872324d P 62d68523dc894c159f9dc7abc1eff8d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 06:04:41.260818 5215 raft_consensus.cc:3037] T 150991c056e34c9d961e0ee0b872324d P 62d68523dc894c159f9dc7abc1eff8d3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:41.261811 5215 raft_consensus.cc:504] T 150991c056e34c9d961e0ee0b872324d P 62d68523dc894c159f9dc7abc1eff8d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } }
I20260808 06:04:41.262113 5215 leader_election.cc:288] T 150991c056e34c9d961e0ee0b872324d P 62d68523dc894c159f9dc7abc1eff8d3 [CANDIDATE]: Term 1 election: Requested vote from peers 65412fe579b64b63b7626d30914841b5 (127.1.151.129:34411), 95ab12dd45364e63a84fbbc433f8ccf0 (127.1.151.131:34917)
I20260808 06:04:41.262488 5094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "150991c056e34c9d961e0ee0b872324d" candidate_uuid: "62d68523dc894c159f9dc7abc1eff8d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95ab12dd45364e63a84fbbc433f8ccf0"
I20260808 06:04:41.262481 4644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "150991c056e34c9d961e0ee0b872324d" candidate_uuid: "62d68523dc894c159f9dc7abc1eff8d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65412fe579b64b63b7626d30914841b5"
I20260808 06:04:41.262600 5094 raft_consensus.cc:3037] T 150991c056e34c9d961e0ee0b872324d P 95ab12dd45364e63a84fbbc433f8ccf0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:41.262639 4644 raft_consensus.cc:3037] T 150991c056e34c9d961e0ee0b872324d P 65412fe579b64b63b7626d30914841b5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:41.263697 5094 raft_consensus.cc:2454] T 150991c056e34c9d961e0ee0b872324d P 95ab12dd45364e63a84fbbc433f8ccf0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62d68523dc894c159f9dc7abc1eff8d3 in term 1.
I20260808 06:04:41.263697 4644 raft_consensus.cc:2454] T 150991c056e34c9d961e0ee0b872324d P 65412fe579b64b63b7626d30914841b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62d68523dc894c159f9dc7abc1eff8d3 in term 1.
I20260808 06:04:41.263967 4753 leader_election.cc:302] T 150991c056e34c9d961e0ee0b872324d P 62d68523dc894c159f9dc7abc1eff8d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62d68523dc894c159f9dc7abc1eff8d3, 95ab12dd45364e63a84fbbc433f8ccf0; no voters:
I20260808 06:04:41.264160 5215 raft_consensus.cc:2781] T 150991c056e34c9d961e0ee0b872324d P 62d68523dc894c159f9dc7abc1eff8d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:04:41.264253 5215 raft_consensus.cc:686] T 150991c056e34c9d961e0ee0b872324d P 62d68523dc894c159f9dc7abc1eff8d3 [term 1 LEADER]: Becoming Leader. State: Replica: 62d68523dc894c159f9dc7abc1eff8d3, State: Running, Role: LEADER
I20260808 06:04:41.264384 5215 consensus_queue.cc:237] T 150991c056e34c9d961e0ee0b872324d P 62d68523dc894c159f9dc7abc1eff8d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } }
I20260808 06:04:41.264991 5217 tablet_replica.cc:439] T 150991c056e34c9d961e0ee0b872324d P 62d68523dc894c159f9dc7abc1eff8d3: TxnStatusTablet state changed. Reason: New leader 62d68523dc894c159f9dc7abc1eff8d3. Latest consensus state: current_term: 1 leader_uuid: "62d68523dc894c159f9dc7abc1eff8d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } } }
I20260808 06:04:41.265107 5217 tablet_replica.cc:442] T 150991c056e34c9d961e0ee0b872324d P 62d68523dc894c159f9dc7abc1eff8d3: This TxnStatusTablet replica's current role is: LEADER
I20260808 06:04:41.265424 5232 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 06:04:41.265661 5232 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 06:04:41.265942 4408 catalog_manager.cc:5495] T 150991c056e34c9d961e0ee0b872324d P 62d68523dc894c159f9dc7abc1eff8d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 62d68523dc894c159f9dc7abc1eff8d3 (127.1.151.130). New cstate: current_term: 1 leader_uuid: "62d68523dc894c159f9dc7abc1eff8d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } health_report { overall_health: UNKNOWN } } }
I20260808 06:04:41.285887 5225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:41.281735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:45285 (local address 127.0.0.1:50576)
0808 06:04:41.281889 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:41.281905 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:41.281997 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:41.282367 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:41.282369 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:41.282377 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 06:04:41.282468 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:41.282473 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.283690 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.283692 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:41.284286 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:41.284295 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.284821 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.284823 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:41.284886 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:41.284931 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 06:04:41.285645 (+ 714us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 06:04:41.285657 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 06:04:41.285707 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 06:04:41.285779 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 06:04:41.285923 5197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:41.281840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50576 (local address 127.1.151.130:45285)
0808 06:04:41.281970 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:41.281975 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:41.281991 (+ 16us) server_negotiation.cc:407] Connection header received
0808 06:04:41.282126 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:41.282129 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:41.282223 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:41.282320 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:04:41.282618 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.283568 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.284393 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.284732 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.284830 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:41.285053 (+ 223us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:04:41.285516 (+ 463us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:04:41.285645 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:41.285736 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 06:04:41.285796 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 06:04:41.290772 4644 raft_consensus.cc:1261] T 150991c056e34c9d961e0ee0b872324d P 65412fe579b64b63b7626d30914841b5 [term 1 FOLLOWER]: Refusing update from remote peer 62d68523dc894c159f9dc7abc1eff8d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:04:41.291414 5217 consensus_queue.cc:1035] T 150991c056e34c9d961e0ee0b872324d P 62d68523dc894c159f9dc7abc1eff8d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:04:41.291535 5094 raft_consensus.cc:1261] T 150991c056e34c9d961e0ee0b872324d P 95ab12dd45364e63a84fbbc433f8ccf0 [term 1 FOLLOWER]: Refusing update from remote peer 62d68523dc894c159f9dc7abc1eff8d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:04:41.292034 5217 consensus_queue.cc:1035] T 150991c056e34c9d961e0ee0b872324d P 62d68523dc894c159f9dc7abc1eff8d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:04:41.295219 5212 tablet_replica.cc:439] T 150991c056e34c9d961e0ee0b872324d P 95ab12dd45364e63a84fbbc433f8ccf0: TxnStatusTablet state changed. Reason: New leader 62d68523dc894c159f9dc7abc1eff8d3. Latest consensus state: current_term: 1 leader_uuid: "62d68523dc894c159f9dc7abc1eff8d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } } }
I20260808 06:04:41.295325 5212 tablet_replica.cc:442] T 150991c056e34c9d961e0ee0b872324d P 95ab12dd45364e63a84fbbc433f8ccf0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 06:04:41.295583 5213 tablet_replica.cc:439] T 150991c056e34c9d961e0ee0b872324d P 65412fe579b64b63b7626d30914841b5: TxnStatusTablet state changed. Reason: New leader 62d68523dc894c159f9dc7abc1eff8d3. Latest consensus state: current_term: 1 leader_uuid: "62d68523dc894c159f9dc7abc1eff8d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } } }
I20260808 06:04:41.295689 5213 tablet_replica.cc:442] T 150991c056e34c9d961e0ee0b872324d P 65412fe579b64b63b7626d30914841b5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 06:04:41.299796 5217 tablet_replica.cc:439] T 150991c056e34c9d961e0ee0b872324d P 62d68523dc894c159f9dc7abc1eff8d3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "62d68523dc894c159f9dc7abc1eff8d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } } }
I20260808 06:04:41.299902 5217 tablet_replica.cc:442] T 150991c056e34c9d961e0ee0b872324d P 62d68523dc894c159f9dc7abc1eff8d3: This TxnStatusTablet replica's current role is: LEADER
I20260808 06:04:41.300062 5235 mvcc.cc:205] Tried to move back new op lower bound from 7316148556960509952 to 7316148556860149760. Current Snapshot: MvccSnapshot[applied={T|T < 7316148556960509952}]
I20260808 06:04:41.300746 5215 tablet_replica.cc:439] T 150991c056e34c9d961e0ee0b872324d P 62d68523dc894c159f9dc7abc1eff8d3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "62d68523dc894c159f9dc7abc1eff8d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } } }
I20260808 06:04:41.300844 5215 tablet_replica.cc:442] T 150991c056e34c9d961e0ee0b872324d P 62d68523dc894c159f9dc7abc1eff8d3: This TxnStatusTablet replica's current role is: LEADER
I20260808 06:04:41.301039 5212 tablet_replica.cc:439] T 150991c056e34c9d961e0ee0b872324d P 95ab12dd45364e63a84fbbc433f8ccf0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "62d68523dc894c159f9dc7abc1eff8d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } } }
I20260808 06:04:41.301138 5212 tablet_replica.cc:442] T 150991c056e34c9d961e0ee0b872324d P 95ab12dd45364e63a84fbbc433f8ccf0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 06:04:41.301842 5213 tablet_replica.cc:439] T 150991c056e34c9d961e0ee0b872324d P 65412fe579b64b63b7626d30914841b5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "62d68523dc894c159f9dc7abc1eff8d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } } }
I20260808 06:04:41.301954 5213 tablet_replica.cc:442] T 150991c056e34c9d961e0ee0b872324d P 65412fe579b64b63b7626d30914841b5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 06:04:41.302881 5238 mvcc.cc:205] Tried to move back new op lower bound from 7316148556960509952 to 7316148556860149760. Current Snapshot: MvccSnapshot[applied={T|T < 7316148556960509952}]
I20260808 06:04:41.303763 5236 mvcc.cc:205] Tried to move back new op lower bound from 7316148556960509952 to 7316148556860149760. Current Snapshot: MvccSnapshot[applied={T|T < 7316148556960509952}]
I20260808 06:04:41.313238 5197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:41.309920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50584 (local address 127.1.151.130:45285)
0808 06:04:41.310042 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:41.310057 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:41.310072 (+ 15us) server_negotiation.cc:407] Connection header received
0808 06:04:41.310202 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:41.310205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:41.310252 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:41.310325 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:04:41.310522 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.311288 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.312119 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.312456 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.312552 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:41.312826 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:04:41.312905 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:04:41.312971 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:41.313034 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 06:04:41.313113 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 06:04:41.323513 5197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:41.320342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:47203 (local address 127.1.151.130:45285)
0808 06:04:41.320443 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:41.320447 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:41.320650 (+ 203us) server_negotiation.cc:407] Connection header received
0808 06:04:41.320719 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:41.320722 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:41.320771 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:41.320827 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:04:41.321058 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.321536 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.322506 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.322719 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.322809 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:41.323084 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:04:41.323143 (+ 59us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:04:41.323199 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:41.323288 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 06:04:41.323327 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 06:04:41.323626 5250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:41.320279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:45285 (local address 127.1.151.130:47203)
0808 06:04:41.320542 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:41.320553 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:41.320646 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:41.320840 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:41.320842 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:41.320849 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 06:04:41.320955 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:41.320979 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.321628 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.321631 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:41.322401 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:41.322411 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.322829 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.322833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:41.322916 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:41.322954 (+ 38us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 06:04:41.323209 (+ 255us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 06:04:41.323215 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 06:04:41.323267 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 06:04:41.323335 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":107,"threads_started":1}
I20260808 06:04:41.329704 4644 raft_consensus.cc:1261] T 19ab362f2b5047b68f894cc446a22414 P 65412fe579b64b63b7626d30914841b5 [term 1 FOLLOWER]: Refusing update from remote peer 62d68523dc894c159f9dc7abc1eff8d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:04:41.329885 5094 raft_consensus.cc:1261] T 19ab362f2b5047b68f894cc446a22414 P 95ab12dd45364e63a84fbbc433f8ccf0 [term 1 FOLLOWER]: Refusing update from remote peer 62d68523dc894c159f9dc7abc1eff8d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:04:41.330274 5215 consensus_queue.cc:1035] T 19ab362f2b5047b68f894cc446a22414 P 62d68523dc894c159f9dc7abc1eff8d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:04:41.330432 5217 consensus_queue.cc:1035] T 19ab362f2b5047b68f894cc446a22414 P 62d68523dc894c159f9dc7abc1eff8d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:04:41.367487 4409 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:04:41.367653 4409 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:04:41.370060 4409 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 43452437015741b889520f1841311734: Sending DeleteTablet for 3 replicas of tablet 19ab362f2b5047b68f894cc446a22414
I20260808 06:04:41.370886 5065 tablet_service.cc:1514] Processing DeleteTablet for tablet 19ab362f2b5047b68f894cc446a22414 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:04:41 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:50340
I20260808 06:04:41.371404 5263 tablet_replica.cc:330] T 19ab362f2b5047b68f894cc446a22414 P 95ab12dd45364e63a84fbbc433f8ccf0: stopping tablet replica
I20260808 06:04:41.371407 4608 tablet_service.cc:1514] Processing DeleteTablet for tablet 19ab362f2b5047b68f894cc446a22414 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:04:41 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:60122
I20260808 06:04:41.371706 5263 raft_consensus.cc:2229] T 19ab362f2b5047b68f894cc446a22414 P 95ab12dd45364e63a84fbbc433f8ccf0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:04:41.371866 4834 tablet_service.cc:1514] Processing DeleteTablet for tablet 19ab362f2b5047b68f894cc446a22414 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:04:41 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:50562
I20260808 06:04:41.371914 5263 raft_consensus.cc:2258] T 19ab362f2b5047b68f894cc446a22414 P 95ab12dd45364e63a84fbbc433f8ccf0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:04:41.371934 5264 tablet_replica.cc:330] T 19ab362f2b5047b68f894cc446a22414 P 65412fe579b64b63b7626d30914841b5: stopping tablet replica
I20260808 06:04:41.372305 5264 raft_consensus.cc:2229] T 19ab362f2b5047b68f894cc446a22414 P 65412fe579b64b63b7626d30914841b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:04:41.372535 5264 raft_consensus.cc:2258] T 19ab362f2b5047b68f894cc446a22414 P 65412fe579b64b63b7626d30914841b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:04:41.372584 4409 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40390:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 06:04:41.372985 4409 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 06:04:41.373646 5263 ts_tablet_manager.cc:1905] T 19ab362f2b5047b68f894cc446a22414 P 95ab12dd45364e63a84fbbc433f8ccf0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:04:41.374256 5264 ts_tablet_manager.cc:1905] T 19ab362f2b5047b68f894cc446a22414 P 65412fe579b64b63b7626d30914841b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:04:41.376828 5263 ts_tablet_manager.cc:1918] T 19ab362f2b5047b68f894cc446a22414 P 95ab12dd45364e63a84fbbc433f8ccf0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 06:04:41.376935 5263 log.cc:1196] T 19ab362f2b5047b68f894cc446a22414 P 95ab12dd45364e63a84fbbc433f8ccf0: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-2/wal/wals/19ab362f2b5047b68f894cc446a22414
I20260808 06:04:41.377137 5264 ts_tablet_manager.cc:1918] T 19ab362f2b5047b68f894cc446a22414 P 65412fe579b64b63b7626d30914841b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 06:04:41.377269 5263 ts_tablet_manager.cc:1939] T 19ab362f2b5047b68f894cc446a22414 P 95ab12dd45364e63a84fbbc433f8ccf0: Deleting consensus metadata
I20260808 06:04:41.377291 5264 log.cc:1196] T 19ab362f2b5047b68f894cc446a22414 P 65412fe579b64b63b7626d30914841b5: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-0/wal/wals/19ab362f2b5047b68f894cc446a22414
I20260808 06:04:41.377660 5264 ts_tablet_manager.cc:1939] T 19ab362f2b5047b68f894cc446a22414 P 65412fe579b64b63b7626d30914841b5: Deleting consensus metadata
I20260808 06:04:41.378242 4608 tablet_service.cc:1467] Processing CreateTablet for tablet 0aa355ee1cf84a43a314fd02564750dd (DEFAULT_TABLE table=test-table [id=8a4781ccc372446ca374850c4e4c70e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:04:41.378501 4608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0aa355ee1cf84a43a314fd02564750dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:41.378774 4379 catalog_manager.cc:4841] TS 95ab12dd45364e63a84fbbc433f8ccf0 (127.1.151.131:34917): tablet 19ab362f2b5047b68f894cc446a22414 (table test-table [id=4b1d132eb62443b78f2c0f7f5ecb3e04]) successfully deleted
I20260808 06:04:41.378829 4380 catalog_manager.cc:4841] TS 65412fe579b64b63b7626d30914841b5 (127.1.151.129:34411): tablet 19ab362f2b5047b68f894cc446a22414 (table test-table [id=4b1d132eb62443b78f2c0f7f5ecb3e04]) successfully deleted
I20260808 06:04:41.380231 5065 tablet_service.cc:1467] Processing CreateTablet for tablet 0aa355ee1cf84a43a314fd02564750dd (DEFAULT_TABLE table=test-table [id=8a4781ccc372446ca374850c4e4c70e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:04:41.380455 5065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0aa355ee1cf84a43a314fd02564750dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:41.380661 5206 tablet_bootstrap.cc:492] T 0aa355ee1cf84a43a314fd02564750dd P 65412fe579b64b63b7626d30914841b5: Bootstrap starting.
I20260808 06:04:41.381593 5206 tablet_bootstrap.cc:654] T 0aa355ee1cf84a43a314fd02564750dd P 65412fe579b64b63b7626d30914841b5: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:41.382417 4834 tablet_service.cc:1467] Processing CreateTablet for tablet 0aa355ee1cf84a43a314fd02564750dd (DEFAULT_TABLE table=test-table [id=8a4781ccc372446ca374850c4e4c70e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:04:41.382689 5202 tablet_bootstrap.cc:492] T 0aa355ee1cf84a43a314fd02564750dd P 95ab12dd45364e63a84fbbc433f8ccf0: Bootstrap starting.
I20260808 06:04:41.382695 4834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0aa355ee1cf84a43a314fd02564750dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:41.382835 5206 tablet_bootstrap.cc:492] T 0aa355ee1cf84a43a314fd02564750dd P 65412fe579b64b63b7626d30914841b5: No bootstrap required, opened a new log
I20260808 06:04:41.382907 5206 ts_tablet_manager.cc:1397] T 0aa355ee1cf84a43a314fd02564750dd P 65412fe579b64b63b7626d30914841b5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 06:04:41.383339 5206 raft_consensus.cc:348] T 0aa355ee1cf84a43a314fd02564750dd P 65412fe579b64b63b7626d30914841b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } }
I20260808 06:04:41.383443 5206 raft_consensus.cc:374] T 0aa355ee1cf84a43a314fd02564750dd P 65412fe579b64b63b7626d30914841b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:41.383472 5206 raft_consensus.cc:729] T 0aa355ee1cf84a43a314fd02564750dd P 65412fe579b64b63b7626d30914841b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65412fe579b64b63b7626d30914841b5, State: Initialized, Role: FOLLOWER
I20260808 06:04:41.383517 5202 tablet_bootstrap.cc:654] T 0aa355ee1cf84a43a314fd02564750dd P 95ab12dd45364e63a84fbbc433f8ccf0: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:41.383589 5206 consensus_queue.cc:260] T 0aa355ee1cf84a43a314fd02564750dd P 65412fe579b64b63b7626d30914841b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } }
I20260808 06:04:41.383842 5206 ts_tablet_manager.cc:1428] T 0aa355ee1cf84a43a314fd02564750dd P 65412fe579b64b63b7626d30914841b5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 06:04:41.384402 5266 tablet_replica.cc:330] T 19ab362f2b5047b68f894cc446a22414 P 62d68523dc894c159f9dc7abc1eff8d3: stopping tablet replica
I20260808 06:04:41.384735 5266 raft_consensus.cc:2229] T 19ab362f2b5047b68f894cc446a22414 P 62d68523dc894c159f9dc7abc1eff8d3 [term 1 LEADER]: Raft consensus shutting down.
I20260808 06:04:41.385129 5266 raft_consensus.cc:2258] T 19ab362f2b5047b68f894cc446a22414 P 62d68523dc894c159f9dc7abc1eff8d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:04:41.385494 5213 raft_consensus.cc:482] T 0aa355ee1cf84a43a314fd02564750dd P 65412fe579b64b63b7626d30914841b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:04:41.385612 5213 raft_consensus.cc:504] T 0aa355ee1cf84a43a314fd02564750dd P 65412fe579b64b63b7626d30914841b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } }
I20260808 06:04:41.386587 5203 tablet_bootstrap.cc:492] T 0aa355ee1cf84a43a314fd02564750dd P 62d68523dc894c159f9dc7abc1eff8d3: Bootstrap starting.
I20260808 06:04:41.386672 5266 ts_tablet_manager.cc:1905] T 19ab362f2b5047b68f894cc446a22414 P 62d68523dc894c159f9dc7abc1eff8d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:04:41.386736 5213 leader_election.cc:288] T 0aa355ee1cf84a43a314fd02564750dd P 65412fe579b64b63b7626d30914841b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62d68523dc894c159f9dc7abc1eff8d3 (127.1.151.130:45285), 95ab12dd45364e63a84fbbc433f8ccf0 (127.1.151.131:34917)
I20260808 06:04:41.387497 5203 tablet_bootstrap.cc:654] T 0aa355ee1cf84a43a314fd02564750dd P 62d68523dc894c159f9dc7abc1eff8d3: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:41.389495 5266 ts_tablet_manager.cc:1918] T 19ab362f2b5047b68f894cc446a22414 P 62d68523dc894c159f9dc7abc1eff8d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 06:04:41.389941 5266 log.cc:1196] T 19ab362f2b5047b68f894cc446a22414 P 62d68523dc894c159f9dc7abc1eff8d3: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-1/wal/wals/19ab362f2b5047b68f894cc446a22414
I20260808 06:04:41.390337 5202 tablet_bootstrap.cc:492] T 0aa355ee1cf84a43a314fd02564750dd P 95ab12dd45364e63a84fbbc433f8ccf0: No bootstrap required, opened a new log
I20260808 06:04:41.390359 5266 ts_tablet_manager.cc:1939] T 19ab362f2b5047b68f894cc446a22414 P 62d68523dc894c159f9dc7abc1eff8d3: Deleting consensus metadata
I20260808 06:04:41.390456 5202 ts_tablet_manager.cc:1397] T 0aa355ee1cf84a43a314fd02564750dd P 95ab12dd45364e63a84fbbc433f8ccf0: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260808 06:04:41.390887 5202 raft_consensus.cc:348] T 0aa355ee1cf84a43a314fd02564750dd P 95ab12dd45364e63a84fbbc433f8ccf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } }
I20260808 06:04:41.391005 5202 raft_consensus.cc:374] T 0aa355ee1cf84a43a314fd02564750dd P 95ab12dd45364e63a84fbbc433f8ccf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:41.391048 5202 raft_consensus.cc:729] T 0aa355ee1cf84a43a314fd02564750dd P 95ab12dd45364e63a84fbbc433f8ccf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95ab12dd45364e63a84fbbc433f8ccf0, State: Initialized, Role: FOLLOWER
I20260808 06:04:41.391189 5202 consensus_queue.cc:260] T 0aa355ee1cf84a43a314fd02564750dd P 95ab12dd45364e63a84fbbc433f8ccf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } }
I20260808 06:04:41.391458 5202 ts_tablet_manager.cc:1428] T 0aa355ee1cf84a43a314fd02564750dd P 95ab12dd45364e63a84fbbc433f8ccf0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 06:04:41.389703 5203 tablet_bootstrap.cc:492] T 0aa355ee1cf84a43a314fd02564750dd P 62d68523dc894c159f9dc7abc1eff8d3: No bootstrap required, opened a new log
I20260808 06:04:41.391654 5203 ts_tablet_manager.cc:1397] T 0aa355ee1cf84a43a314fd02564750dd P 62d68523dc894c159f9dc7abc1eff8d3: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 06:04:41.391656 4380 catalog_manager.cc:4841] TS 62d68523dc894c159f9dc7abc1eff8d3 (127.1.151.130:45285): tablet 19ab362f2b5047b68f894cc446a22414 (table test-table [id=4b1d132eb62443b78f2c0f7f5ecb3e04]) successfully deleted
I20260808 06:04:41.392136 5203 raft_consensus.cc:348] T 0aa355ee1cf84a43a314fd02564750dd P 62d68523dc894c159f9dc7abc1eff8d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } }
I20260808 06:04:41.392247 5203 raft_consensus.cc:374] T 0aa355ee1cf84a43a314fd02564750dd P 62d68523dc894c159f9dc7abc1eff8d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:41.392288 5203 raft_consensus.cc:729] T 0aa355ee1cf84a43a314fd02564750dd P 62d68523dc894c159f9dc7abc1eff8d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62d68523dc894c159f9dc7abc1eff8d3, State: Initialized, Role: FOLLOWER
I20260808 06:04:41.392407 5203 consensus_queue.cc:260] T 0aa355ee1cf84a43a314fd02564750dd P 62d68523dc894c159f9dc7abc1eff8d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } }
I20260808 06:04:41.392665 5203 ts_tablet_manager.cc:1428] T 0aa355ee1cf84a43a314fd02564750dd P 62d68523dc894c159f9dc7abc1eff8d3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 06:04:41.395768 5195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:41.387178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:44091 (local address 127.1.151.131:34917)
0808 06:04:41.387295 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:41.387300 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:41.388581 (+ 1281us) server_negotiation.cc:407] Connection header received
0808 06:04:41.388775 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:41.388778 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:41.388825 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:41.388890 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:04:41.389377 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.390142 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.394521 (+ 4379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.395118 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.395195 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:41.395233 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:41.395573 (+ 340us) server_negotiation.cc:299] Negotiation successful
0808 06:04:41.395654 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 06:04:41.395774 5275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:41.387137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:34917 (local address 127.1.151.129:44091)
0808 06:04:41.388492 (+ 1355us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:41.388507 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:41.388653 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:41.389100 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:41.389103 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:41.389124 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:04:41.389248 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:41.389254 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.392004 (+ 2750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.392008 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:41.393530 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:41.393541 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.395316 (+ 1775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.395319 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:41.395425 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:41.395455 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 06:04:41.395528 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 06:04:41.395602 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1290,"mutex_wait_us":4,"thread_start_us":67,"threads_started":1}
I20260808 06:04:41.396322 5094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0aa355ee1cf84a43a314fd02564750dd" candidate_uuid: "65412fe579b64b63b7626d30914841b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" is_pre_election: true
I20260808 06:04:41.396459 5094 raft_consensus.cc:2454] T 0aa355ee1cf84a43a314fd02564750dd P 95ab12dd45364e63a84fbbc433f8ccf0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65412fe579b64b63b7626d30914841b5 in term 0.
I20260808 06:04:41.396804 4535 leader_election.cc:302] T 0aa355ee1cf84a43a314fd02564750dd P 65412fe579b64b63b7626d30914841b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65412fe579b64b63b7626d30914841b5, 95ab12dd45364e63a84fbbc433f8ccf0; no voters:
I20260808 06:04:41.397066 5213 raft_consensus.cc:2781] T 0aa355ee1cf84a43a314fd02564750dd P 65412fe579b64b63b7626d30914841b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 06:04:41.397140 5213 raft_consensus.cc:482] T 0aa355ee1cf84a43a314fd02564750dd P 65412fe579b64b63b7626d30914841b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 06:04:41.397181 5213 raft_consensus.cc:3037] T 0aa355ee1cf84a43a314fd02564750dd P 65412fe579b64b63b7626d30914841b5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:41.397758 5197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:41.387895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:44479 (local address 127.1.151.130:45285)
0808 06:04:41.388025 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:41.388029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:41.393792 (+ 5763us) server_negotiation.cc:407] Connection header received
0808 06:04:41.393969 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:41.393972 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:41.394024 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:41.394072 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:04:41.394488 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.395187 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.396552 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.397103 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.397195 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:41.397244 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:41.397578 (+ 334us) server_negotiation.cc:299] Negotiation successful
0808 06:04:41.397652 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 06:04:41.398082 5273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:41.387495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:45285 (local address 127.1.151.129:44479)
0808 06:04:41.393714 (+ 6219us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:41.393728 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:41.393861 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:41.394224 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:41.394227 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:41.394243 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:04:41.394361 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:41.394367 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.395305 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.395308 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:41.396418 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:41.396428 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.397336 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.397340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:41.397441 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:41.397463 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 06:04:41.397874 (+ 411us) client_negotiation.cc:240] Negotiation successful
0808 06:04:41.397944 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5456,"thread_start_us":98,"threads_started":1}
I20260808 06:04:41.398351 5213 raft_consensus.cc:504] T 0aa355ee1cf84a43a314fd02564750dd P 65412fe579b64b63b7626d30914841b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } }
I20260808 06:04:41.398591 4863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0aa355ee1cf84a43a314fd02564750dd" candidate_uuid: "65412fe579b64b63b7626d30914841b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62d68523dc894c159f9dc7abc1eff8d3" is_pre_election: true
I20260808 06:04:41.398729 5213 leader_election.cc:288] T 0aa355ee1cf84a43a314fd02564750dd P 65412fe579b64b63b7626d30914841b5 [CANDIDATE]: Term 1 election: Requested vote from peers 62d68523dc894c159f9dc7abc1eff8d3 (127.1.151.130:45285), 95ab12dd45364e63a84fbbc433f8ccf0 (127.1.151.131:34917)
I20260808 06:04:41.398777 4863 raft_consensus.cc:2454] T 0aa355ee1cf84a43a314fd02564750dd P 62d68523dc894c159f9dc7abc1eff8d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65412fe579b64b63b7626d30914841b5 in term 0.
I20260808 06:04:41.399042 4862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0aa355ee1cf84a43a314fd02564750dd" candidate_uuid: "65412fe579b64b63b7626d30914841b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62d68523dc894c159f9dc7abc1eff8d3"
I20260808 06:04:41.399142 4862 raft_consensus.cc:3037] T 0aa355ee1cf84a43a314fd02564750dd P 62d68523dc894c159f9dc7abc1eff8d3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:41.399571 5094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0aa355ee1cf84a43a314fd02564750dd" candidate_uuid: "65412fe579b64b63b7626d30914841b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95ab12dd45364e63a84fbbc433f8ccf0"
I20260808 06:04:41.399691 5094 raft_consensus.cc:3037] T 0aa355ee1cf84a43a314fd02564750dd P 95ab12dd45364e63a84fbbc433f8ccf0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:41.400092 4862 raft_consensus.cc:2454] T 0aa355ee1cf84a43a314fd02564750dd P 62d68523dc894c159f9dc7abc1eff8d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65412fe579b64b63b7626d30914841b5 in term 1.
I20260808 06:04:41.400367 4539 leader_election.cc:302] T 0aa355ee1cf84a43a314fd02564750dd P 65412fe579b64b63b7626d30914841b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62d68523dc894c159f9dc7abc1eff8d3, 65412fe579b64b63b7626d30914841b5; no voters:
I20260808 06:04:41.400535 5213 raft_consensus.cc:2781] T 0aa355ee1cf84a43a314fd02564750dd P 65412fe579b64b63b7626d30914841b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:04:41.400719 5213 raft_consensus.cc:686] T 0aa355ee1cf84a43a314fd02564750dd P 65412fe579b64b63b7626d30914841b5 [term 1 LEADER]: Becoming Leader. State: Replica: 65412fe579b64b63b7626d30914841b5, State: Running, Role: LEADER
I20260808 06:04:41.400851 5094 raft_consensus.cc:2454] T 0aa355ee1cf84a43a314fd02564750dd P 95ab12dd45364e63a84fbbc433f8ccf0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65412fe579b64b63b7626d30914841b5 in term 1.
I20260808 06:04:41.400947 5213 consensus_queue.cc:237] T 0aa355ee1cf84a43a314fd02564750dd P 65412fe579b64b63b7626d30914841b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } }
I20260808 06:04:41.403087 4408 catalog_manager.cc:5495] T 0aa355ee1cf84a43a314fd02564750dd P 65412fe579b64b63b7626d30914841b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 65412fe579b64b63b7626d30914841b5 (127.1.151.129). New cstate: current_term: 1 leader_uuid: "65412fe579b64b63b7626d30914841b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } health_report { overall_health: UNKNOWN } } }
I20260808 06:04:41.418506 5199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:41.415139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60132 (local address 127.1.151.129:34411)
0808 06:04:41.415238 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:41.415243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:41.415297 (+ 54us) server_negotiation.cc:407] Connection header received
0808 06:04:41.415468 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:41.415471 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:41.415525 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:41.415626 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:04:41.415842 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.416343 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.417127 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.417358 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.417443 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:41.417693 (+ 250us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:04:41.418129 (+ 436us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:04:41.418276 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:41.418313 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 06:04:41.418369 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 06:04:41.429407 5197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:41.426214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:35187 (local address 127.1.151.130:45285)
0808 06:04:41.426335 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:41.426338 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:41.426472 (+ 134us) server_negotiation.cc:407] Connection header received
0808 06:04:41.426632 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:41.426635 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:41.426678 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:41.426737 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:04:41.427033 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.427511 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.428354 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.428616 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.428706 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:41.428998 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:04:41.429068 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:04:41.429126 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:41.429219 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 06:04:41.429268 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 06:04:41.429430 5282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:41.426040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:45285 (local address 127.1.151.129:35187)
0808 06:04:41.426389 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:41.426403 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:41.426524 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:41.426770 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:41.426774 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:41.426785 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 06:04:41.426920 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:41.426926 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.427631 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.427634 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:41.428246 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:41.428255 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.428743 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.428747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:41.428834 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:41.428877 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 06:04:41.429144 (+ 267us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 06:04:41.429151 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 06:04:41.429209 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 06:04:41.429291 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":143,"threads_started":1}
I20260808 06:04:41.435316 5094 raft_consensus.cc:1261] T 0aa355ee1cf84a43a314fd02564750dd P 95ab12dd45364e63a84fbbc433f8ccf0 [term 1 FOLLOWER]: Refusing update from remote peer 65412fe579b64b63b7626d30914841b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:04:41.435425 4862 raft_consensus.cc:1261] T 0aa355ee1cf84a43a314fd02564750dd P 62d68523dc894c159f9dc7abc1eff8d3 [term 1 FOLLOWER]: Refusing update from remote peer 65412fe579b64b63b7626d30914841b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:04:41.435860 5278 consensus_queue.cc:1035] T 0aa355ee1cf84a43a314fd02564750dd P 65412fe579b64b63b7626d30914841b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:04:41.436247 5213 consensus_queue.cc:1035] T 0aa355ee1cf84a43a314fd02564750dd P 65412fe579b64b63b7626d30914841b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:04:41.455999 5199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:41.452408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:36627 (local address 127.1.151.129:34411)
0808 06:04:41.452716 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:41.452720 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:41.452733 (+ 13us) server_negotiation.cc:407] Connection header received
0808 06:04:41.452774 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:41.452777 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:41.452823 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:41.452892 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:04:41.453208 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.453995 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.454882 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.455232 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.455329 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:41.455600 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:04:41.455685 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:04:41.455759 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:41.455852 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 06:04:41.455900 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235}
I20260808 06:04:41.456071 5250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:41.451957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:34411 (local address 127.1.151.130:36627)
0808 06:04:41.452197 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:41.452209 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:41.452279 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:41.452996 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:41.452998 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:41.453006 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 06:04:41.453093 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:41.453099 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.454109 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.454112 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:41.454767 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:41.454779 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:41.455345 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:41.455349 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:41.455434 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:41.455470 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 06:04:41.455769 (+ 299us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 06:04:41.455777 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 06:04:41.455844 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 06:04:41.455928 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157}
W20260808 06:04:41.456732 5236 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 06:04:43.016077 4408 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:04:43.016234 4408 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:04:43.018448 4408 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 43452437015741b889520f1841311734: Sending DeleteTablet for 3 replicas of tablet 0aa355ee1cf84a43a314fd02564750dd
I20260808 06:04:43.018906 4608 tablet_service.cc:1514] Processing DeleteTablet for tablet 0aa355ee1cf84a43a314fd02564750dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:04:43 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:60122
I20260808 06:04:43.019025 5065 tablet_service.cc:1514] Processing DeleteTablet for tablet 0aa355ee1cf84a43a314fd02564750dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:04:43 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:50340
I20260808 06:04:43.019217 4835 tablet_service.cc:1514] Processing DeleteTablet for tablet 0aa355ee1cf84a43a314fd02564750dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:04:43 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:50562
I20260808 06:04:43.019914 5324 tablet_replica.cc:330] T 0aa355ee1cf84a43a314fd02564750dd P 62d68523dc894c159f9dc7abc1eff8d3: stopping tablet replica
I20260808 06:04:43.019943 5322 tablet_replica.cc:330] T 0aa355ee1cf84a43a314fd02564750dd P 95ab12dd45364e63a84fbbc433f8ccf0: stopping tablet replica
I20260808 06:04:43.020059 5324 raft_consensus.cc:2229] T 0aa355ee1cf84a43a314fd02564750dd P 62d68523dc894c159f9dc7abc1eff8d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:04:43.020191 5324 raft_consensus.cc:2258] T 0aa355ee1cf84a43a314fd02564750dd P 62d68523dc894c159f9dc7abc1eff8d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:04:43.020182 5322 raft_consensus.cc:2229] T 0aa355ee1cf84a43a314fd02564750dd P 95ab12dd45364e63a84fbbc433f8ccf0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:04:43.020347 5322 raft_consensus.cc:2258] T 0aa355ee1cf84a43a314fd02564750dd P 95ab12dd45364e63a84fbbc433f8ccf0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:04:43.020953 5322 ts_tablet_manager.cc:1905] T 0aa355ee1cf84a43a314fd02564750dd P 95ab12dd45364e63a84fbbc433f8ccf0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:04:43.020987 5324 ts_tablet_manager.cc:1905] T 0aa355ee1cf84a43a314fd02564750dd P 62d68523dc894c159f9dc7abc1eff8d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:04:43.021041 5321 tablet_replica.cc:330] T 0aa355ee1cf84a43a314fd02564750dd P 65412fe579b64b63b7626d30914841b5: stopping tablet replica
I20260808 06:04:43.021178 5321 raft_consensus.cc:2229] T 0aa355ee1cf84a43a314fd02564750dd P 65412fe579b64b63b7626d30914841b5 [term 1 LEADER]: Raft consensus shutting down.
I20260808 06:04:43.021484 5321 raft_consensus.cc:2258] T 0aa355ee1cf84a43a314fd02564750dd P 65412fe579b64b63b7626d30914841b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:04:43.022143 5321 ts_tablet_manager.cc:1905] T 0aa355ee1cf84a43a314fd02564750dd P 65412fe579b64b63b7626d30914841b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:04:43.023734 5324 ts_tablet_manager.cc:1918] T 0aa355ee1cf84a43a314fd02564750dd P 62d68523dc894c159f9dc7abc1eff8d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 06:04:43.023788 5322 ts_tablet_manager.cc:1918] T 0aa355ee1cf84a43a314fd02564750dd P 95ab12dd45364e63a84fbbc433f8ccf0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 06:04:43.023819 5324 log.cc:1196] T 0aa355ee1cf84a43a314fd02564750dd P 62d68523dc894c159f9dc7abc1eff8d3: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-1/wal/wals/0aa355ee1cf84a43a314fd02564750dd
I20260808 06:04:43.023845 5322 log.cc:1196] T 0aa355ee1cf84a43a314fd02564750dd P 95ab12dd45364e63a84fbbc433f8ccf0: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-2/wal/wals/0aa355ee1cf84a43a314fd02564750dd
I20260808 06:04:43.024070 5324 ts_tablet_manager.cc:1939] T 0aa355ee1cf84a43a314fd02564750dd P 62d68523dc894c159f9dc7abc1eff8d3: Deleting consensus metadata
I20260808 06:04:43.024113 5322 ts_tablet_manager.cc:1939] T 0aa355ee1cf84a43a314fd02564750dd P 95ab12dd45364e63a84fbbc433f8ccf0: Deleting consensus metadata
I20260808 06:04:43.024264 5321 ts_tablet_manager.cc:1918] T 0aa355ee1cf84a43a314fd02564750dd P 65412fe579b64b63b7626d30914841b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 06:04:43.024331 5321 log.cc:1196] T 0aa355ee1cf84a43a314fd02564750dd P 65412fe579b64b63b7626d30914841b5: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-0/wal/wals/0aa355ee1cf84a43a314fd02564750dd
I20260808 06:04:43.024560 5321 ts_tablet_manager.cc:1939] T 0aa355ee1cf84a43a314fd02564750dd P 65412fe579b64b63b7626d30914841b5: Deleting consensus metadata
I20260808 06:04:43.024753 4380 catalog_manager.cc:4841] TS 62d68523dc894c159f9dc7abc1eff8d3 (127.1.151.130:45285): tablet 0aa355ee1cf84a43a314fd02564750dd (table test-table [id=8a4781ccc372446ca374850c4e4c70e4]) successfully deleted
I20260808 06:04:43.024924 4379 catalog_manager.cc:4841] TS 95ab12dd45364e63a84fbbc433f8ccf0 (127.1.151.131:34917): tablet 0aa355ee1cf84a43a314fd02564750dd (table test-table [id=8a4781ccc372446ca374850c4e4c70e4]) successfully deleted
I20260808 06:04:43.025195 4380 catalog_manager.cc:4841] TS 65412fe579b64b63b7626d30914841b5 (127.1.151.129:34411): tablet 0aa355ee1cf84a43a314fd02564750dd (table test-table [id=8a4781ccc372446ca374850c4e4c70e4]) successfully deleted
Aug 08 06:04:43 dist-test-slave-zj8q krb5kdc[4339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169083, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 06:04:43 dist-test-slave-zj8q krb5kdc[4339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169083, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:04:43.048527 5336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:43.040788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55410 (local address 127.1.151.190:34837)
0808 06:04:43.041070 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:43.041078 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:43.041093 (+ 15us) server_negotiation.cc:407] Connection header received
0808 06:04:43.041240 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:43.041244 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:43.041293 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:43.041366 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:43.042032 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:43.042645 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:43.043452 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:43.043688 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:43.043773 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:43.045999 (+ 2226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:43.046002 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:43.046004 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:43.046027 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:43.047279 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:43.047624 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:43.047627 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:43.047628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:43.047694 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:43.047869 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:43.047871 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:43.047872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:43.048024 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:43.048133 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:43.048283 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 06:04:43.048380 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":101,"threads_started":1}
I20260808 06:04:43.050966 4408 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55410:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 06:04:43.051404 4408 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 06:04:43.055832 4608 tablet_service.cc:1467] Processing CreateTablet for tablet 85d3a2f8d4d944ac94565323c63a0b84 (DEFAULT_TABLE table=test-table [id=4f30c9edcb954d8dbf5458c9bf3974aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:04:43.055913 4835 tablet_service.cc:1467] Processing CreateTablet for tablet 85d3a2f8d4d944ac94565323c63a0b84 (DEFAULT_TABLE table=test-table [id=4f30c9edcb954d8dbf5458c9bf3974aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:04:43.055938 5065 tablet_service.cc:1467] Processing CreateTablet for tablet 85d3a2f8d4d944ac94565323c63a0b84 (DEFAULT_TABLE table=test-table [id=4f30c9edcb954d8dbf5458c9bf3974aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:04:43.056133 4608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85d3a2f8d4d944ac94565323c63a0b84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:43.056169 5065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85d3a2f8d4d944ac94565323c63a0b84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:43.056233 4835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85d3a2f8d4d944ac94565323c63a0b84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:43.058907 5337 tablet_bootstrap.cc:492] T 85d3a2f8d4d944ac94565323c63a0b84 P 62d68523dc894c159f9dc7abc1eff8d3: Bootstrap starting.
I20260808 06:04:43.059051 5338 tablet_bootstrap.cc:492] T 85d3a2f8d4d944ac94565323c63a0b84 P 95ab12dd45364e63a84fbbc433f8ccf0: Bootstrap starting.
I20260808 06:04:43.059851 5338 tablet_bootstrap.cc:654] T 85d3a2f8d4d944ac94565323c63a0b84 P 95ab12dd45364e63a84fbbc433f8ccf0: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:43.059851 5337 tablet_bootstrap.cc:654] T 85d3a2f8d4d944ac94565323c63a0b84 P 62d68523dc894c159f9dc7abc1eff8d3: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:43.060494 5339 tablet_bootstrap.cc:492] T 85d3a2f8d4d944ac94565323c63a0b84 P 65412fe579b64b63b7626d30914841b5: Bootstrap starting.
I20260808 06:04:43.061056 5338 tablet_bootstrap.cc:492] T 85d3a2f8d4d944ac94565323c63a0b84 P 95ab12dd45364e63a84fbbc433f8ccf0: No bootstrap required, opened a new log
I20260808 06:04:43.061148 5338 ts_tablet_manager.cc:1397] T 85d3a2f8d4d944ac94565323c63a0b84 P 95ab12dd45364e63a84fbbc433f8ccf0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 06:04:43.061168 5337 tablet_bootstrap.cc:492] T 85d3a2f8d4d944ac94565323c63a0b84 P 62d68523dc894c159f9dc7abc1eff8d3: No bootstrap required, opened a new log
I20260808 06:04:43.061254 5337 ts_tablet_manager.cc:1397] T 85d3a2f8d4d944ac94565323c63a0b84 P 62d68523dc894c159f9dc7abc1eff8d3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 06:04:43.061336 5339 tablet_bootstrap.cc:654] T 85d3a2f8d4d944ac94565323c63a0b84 P 65412fe579b64b63b7626d30914841b5: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:43.061589 5338 raft_consensus.cc:348] T 85d3a2f8d4d944ac94565323c63a0b84 P 95ab12dd45364e63a84fbbc433f8ccf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } }
I20260808 06:04:43.061722 5338 raft_consensus.cc:374] T 85d3a2f8d4d944ac94565323c63a0b84 P 95ab12dd45364e63a84fbbc433f8ccf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:43.061758 5338 raft_consensus.cc:729] T 85d3a2f8d4d944ac94565323c63a0b84 P 95ab12dd45364e63a84fbbc433f8ccf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95ab12dd45364e63a84fbbc433f8ccf0, State: Initialized, Role: FOLLOWER
I20260808 06:04:43.061828 5337 raft_consensus.cc:348] T 85d3a2f8d4d944ac94565323c63a0b84 P 62d68523dc894c159f9dc7abc1eff8d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } }
I20260808 06:04:43.061944 5337 raft_consensus.cc:374] T 85d3a2f8d4d944ac94565323c63a0b84 P 62d68523dc894c159f9dc7abc1eff8d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:43.061908 5338 consensus_queue.cc:260] T 85d3a2f8d4d944ac94565323c63a0b84 P 95ab12dd45364e63a84fbbc433f8ccf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } }
I20260808 06:04:43.061980 5337 raft_consensus.cc:729] T 85d3a2f8d4d944ac94565323c63a0b84 P 62d68523dc894c159f9dc7abc1eff8d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62d68523dc894c159f9dc7abc1eff8d3, State: Initialized, Role: FOLLOWER
I20260808 06:04:43.062114 5337 consensus_queue.cc:260] T 85d3a2f8d4d944ac94565323c63a0b84 P 62d68523dc894c159f9dc7abc1eff8d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } }
I20260808 06:04:43.062371 5338 ts_tablet_manager.cc:1428] T 85d3a2f8d4d944ac94565323c63a0b84 P 95ab12dd45364e63a84fbbc433f8ccf0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 06:04:43.062713 5339 tablet_bootstrap.cc:492] T 85d3a2f8d4d944ac94565323c63a0b84 P 65412fe579b64b63b7626d30914841b5: No bootstrap required, opened a new log
I20260808 06:04:43.062814 5339 ts_tablet_manager.cc:1397] T 85d3a2f8d4d944ac94565323c63a0b84 P 65412fe579b64b63b7626d30914841b5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 06:04:43.062844 5337 ts_tablet_manager.cc:1428] T 85d3a2f8d4d944ac94565323c63a0b84 P 62d68523dc894c159f9dc7abc1eff8d3: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 06:04:43.063215 5339 raft_consensus.cc:348] T 85d3a2f8d4d944ac94565323c63a0b84 P 65412fe579b64b63b7626d30914841b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } }
I20260808 06:04:43.063308 5339 raft_consensus.cc:374] T 85d3a2f8d4d944ac94565323c63a0b84 P 65412fe579b64b63b7626d30914841b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:43.063338 5339 raft_consensus.cc:729] T 85d3a2f8d4d944ac94565323c63a0b84 P 65412fe579b64b63b7626d30914841b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65412fe579b64b63b7626d30914841b5, State: Initialized, Role: FOLLOWER
I20260808 06:04:43.063459 5339 consensus_queue.cc:260] T 85d3a2f8d4d944ac94565323c63a0b84 P 65412fe579b64b63b7626d30914841b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } }
I20260808 06:04:43.063709 5339 ts_tablet_manager.cc:1428] T 85d3a2f8d4d944ac94565323c63a0b84 P 65412fe579b64b63b7626d30914841b5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 06:04:43.080741 5313 raft_consensus.cc:482] T 85d3a2f8d4d944ac94565323c63a0b84 P 65412fe579b64b63b7626d30914841b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:04:43.080834 5313 raft_consensus.cc:504] T 85d3a2f8d4d944ac94565323c63a0b84 P 65412fe579b64b63b7626d30914841b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } }
I20260808 06:04:43.081207 5313 leader_election.cc:288] T 85d3a2f8d4d944ac94565323c63a0b84 P 65412fe579b64b63b7626d30914841b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62d68523dc894c159f9dc7abc1eff8d3 (127.1.151.130:45285), 95ab12dd45364e63a84fbbc433f8ccf0 (127.1.151.131:34917)
I20260808 06:04:43.081574 5094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85d3a2f8d4d944ac94565323c63a0b84" candidate_uuid: "65412fe579b64b63b7626d30914841b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" is_pre_election: true
I20260808 06:04:43.081553 4863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85d3a2f8d4d944ac94565323c63a0b84" candidate_uuid: "65412fe579b64b63b7626d30914841b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62d68523dc894c159f9dc7abc1eff8d3" is_pre_election: true
I20260808 06:04:43.081699 5094 raft_consensus.cc:2454] T 85d3a2f8d4d944ac94565323c63a0b84 P 95ab12dd45364e63a84fbbc433f8ccf0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65412fe579b64b63b7626d30914841b5 in term 0.
I20260808 06:04:43.081758 4863 raft_consensus.cc:2454] T 85d3a2f8d4d944ac94565323c63a0b84 P 62d68523dc894c159f9dc7abc1eff8d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65412fe579b64b63b7626d30914841b5 in term 0.
I20260808 06:04:43.081991 4535 leader_election.cc:302] T 85d3a2f8d4d944ac94565323c63a0b84 P 65412fe579b64b63b7626d30914841b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65412fe579b64b63b7626d30914841b5, 95ab12dd45364e63a84fbbc433f8ccf0; no voters:
I20260808 06:04:43.082203 5313 raft_consensus.cc:2781] T 85d3a2f8d4d944ac94565323c63a0b84 P 65412fe579b64b63b7626d30914841b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 06:04:43.082263 5313 raft_consensus.cc:482] T 85d3a2f8d4d944ac94565323c63a0b84 P 65412fe579b64b63b7626d30914841b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 06:04:43.082302 5313 raft_consensus.cc:3037] T 85d3a2f8d4d944ac94565323c63a0b84 P 65412fe579b64b63b7626d30914841b5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:43.083230 5313 raft_consensus.cc:504] T 85d3a2f8d4d944ac94565323c63a0b84 P 65412fe579b64b63b7626d30914841b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } }
I20260808 06:04:43.083498 5313 leader_election.cc:288] T 85d3a2f8d4d944ac94565323c63a0b84 P 65412fe579b64b63b7626d30914841b5 [CANDIDATE]: Term 1 election: Requested vote from peers 62d68523dc894c159f9dc7abc1eff8d3 (127.1.151.130:45285), 95ab12dd45364e63a84fbbc433f8ccf0 (127.1.151.131:34917)
I20260808 06:04:43.083786 4863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85d3a2f8d4d944ac94565323c63a0b84" candidate_uuid: "65412fe579b64b63b7626d30914841b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62d68523dc894c159f9dc7abc1eff8d3"
I20260808 06:04:43.083806 5094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85d3a2f8d4d944ac94565323c63a0b84" candidate_uuid: "65412fe579b64b63b7626d30914841b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95ab12dd45364e63a84fbbc433f8ccf0"
I20260808 06:04:43.083878 4863 raft_consensus.cc:3037] T 85d3a2f8d4d944ac94565323c63a0b84 P 62d68523dc894c159f9dc7abc1eff8d3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:43.083891 5094 raft_consensus.cc:3037] T 85d3a2f8d4d944ac94565323c63a0b84 P 95ab12dd45364e63a84fbbc433f8ccf0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:43.084645 5094 raft_consensus.cc:2454] T 85d3a2f8d4d944ac94565323c63a0b84 P 95ab12dd45364e63a84fbbc433f8ccf0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65412fe579b64b63b7626d30914841b5 in term 1.
I20260808 06:04:43.084766 4863 raft_consensus.cc:2454] T 85d3a2f8d4d944ac94565323c63a0b84 P 62d68523dc894c159f9dc7abc1eff8d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65412fe579b64b63b7626d30914841b5 in term 1.
I20260808 06:04:43.084913 4535 leader_election.cc:302] T 85d3a2f8d4d944ac94565323c63a0b84 P 65412fe579b64b63b7626d30914841b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65412fe579b64b63b7626d30914841b5, 95ab12dd45364e63a84fbbc433f8ccf0; no voters:
I20260808 06:04:43.085141 5313 raft_consensus.cc:2781] T 85d3a2f8d4d944ac94565323c63a0b84 P 65412fe579b64b63b7626d30914841b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:04:43.085219 5313 raft_consensus.cc:686] T 85d3a2f8d4d944ac94565323c63a0b84 P 65412fe579b64b63b7626d30914841b5 [term 1 LEADER]: Becoming Leader. State: Replica: 65412fe579b64b63b7626d30914841b5, State: Running, Role: LEADER
I20260808 06:04:43.085346 5313 consensus_queue.cc:237] T 85d3a2f8d4d944ac94565323c63a0b84 P 65412fe579b64b63b7626d30914841b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } }
I20260808 06:04:43.086844 4406 catalog_manager.cc:5495] T 85d3a2f8d4d944ac94565323c63a0b84 P 65412fe579b64b63b7626d30914841b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 65412fe579b64b63b7626d30914841b5 (127.1.151.129). New cstate: current_term: 1 leader_uuid: "65412fe579b64b63b7626d30914841b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } health_report { overall_health: UNKNOWN } } }
I20260808 06:04:43.105461 5348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:43.102433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37990 (local address 127.1.151.129:34411)
0808 06:04:43.102706 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:43.102713 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:43.102724 (+ 11us) server_negotiation.cc:407] Connection header received
0808 06:04:43.102763 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:43.102765 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:43.102803 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:43.102868 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:04:43.103082 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:43.103598 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:43.104455 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:43.104716 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:43.104812 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:43.104999 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:04:43.105067 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:04:43.105117 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:43.105249 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 06:04:43.105295 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":100,"threads_started":1}
I20260808 06:04:43.110980 4863 raft_consensus.cc:1261] T 85d3a2f8d4d944ac94565323c63a0b84 P 62d68523dc894c159f9dc7abc1eff8d3 [term 1 FOLLOWER]: Refusing update from remote peer 65412fe579b64b63b7626d30914841b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:04:43.111029 5094 raft_consensus.cc:1261] T 85d3a2f8d4d944ac94565323c63a0b84 P 95ab12dd45364e63a84fbbc433f8ccf0 [term 1 FOLLOWER]: Refusing update from remote peer 65412fe579b64b63b7626d30914841b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:04:43.111363 5313 consensus_queue.cc:1035] T 85d3a2f8d4d944ac94565323c63a0b84 P 65412fe579b64b63b7626d30914841b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:04:43.111548 5292 consensus_queue.cc:1035] T 85d3a2f8d4d944ac94565323c63a0b84 P 65412fe579b64b63b7626d30914841b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:04:43.140538 4406 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:04:43.140688 4406 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:04:43.142494 4406 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 43452437015741b889520f1841311734: Sending DeleteTablet for 3 replicas of tablet 85d3a2f8d4d944ac94565323c63a0b84
I20260808 06:04:43.142979 4608 tablet_service.cc:1514] Processing DeleteTablet for tablet 85d3a2f8d4d944ac94565323c63a0b84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:04:43 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:60122
I20260808 06:04:43.143005 4835 tablet_service.cc:1514] Processing DeleteTablet for tablet 85d3a2f8d4d944ac94565323c63a0b84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:04:43 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:50562
I20260808 06:04:43.143042 5065 tablet_service.cc:1514] Processing DeleteTablet for tablet 85d3a2f8d4d944ac94565323c63a0b84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:04:43 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:50340
I20260808 06:04:43.143155 5321 tablet_replica.cc:330] T 85d3a2f8d4d944ac94565323c63a0b84 P 65412fe579b64b63b7626d30914841b5: stopping tablet replica
I20260808 06:04:43.143189 5324 tablet_replica.cc:330] T 85d3a2f8d4d944ac94565323c63a0b84 P 62d68523dc894c159f9dc7abc1eff8d3: stopping tablet replica
I20260808 06:04:43.143252 5321 raft_consensus.cc:2229] T 85d3a2f8d4d944ac94565323c63a0b84 P 65412fe579b64b63b7626d30914841b5 [term 1 LEADER]: Raft consensus shutting down.
I20260808 06:04:43.143282 5324 raft_consensus.cc:2229] T 85d3a2f8d4d944ac94565323c63a0b84 P 62d68523dc894c159f9dc7abc1eff8d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:04:43.143357 5324 raft_consensus.cc:2258] T 85d3a2f8d4d944ac94565323c63a0b84 P 62d68523dc894c159f9dc7abc1eff8d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:04:43.143379 5321 raft_consensus.cc:2258] T 85d3a2f8d4d944ac94565323c63a0b84 P 65412fe579b64b63b7626d30914841b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:04:43.143494 5322 tablet_replica.cc:330] T 85d3a2f8d4d944ac94565323c63a0b84 P 95ab12dd45364e63a84fbbc433f8ccf0: stopping tablet replica
I20260808 06:04:43.143647 5322 raft_consensus.cc:2229] T 85d3a2f8d4d944ac94565323c63a0b84 P 95ab12dd45364e63a84fbbc433f8ccf0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:04:43.143730 5322 raft_consensus.cc:2258] T 85d3a2f8d4d944ac94565323c63a0b84 P 95ab12dd45364e63a84fbbc433f8ccf0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:04:43.143932 5321 ts_tablet_manager.cc:1905] T 85d3a2f8d4d944ac94565323c63a0b84 P 65412fe579b64b63b7626d30914841b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:04:43.144382 5324 ts_tablet_manager.cc:1905] T 85d3a2f8d4d944ac94565323c63a0b84 P 62d68523dc894c159f9dc7abc1eff8d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:04:43.144595 5322 ts_tablet_manager.cc:1905] T 85d3a2f8d4d944ac94565323c63a0b84 P 95ab12dd45364e63a84fbbc433f8ccf0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:04:43.146586 5321 ts_tablet_manager.cc:1918] T 85d3a2f8d4d944ac94565323c63a0b84 P 65412fe579b64b63b7626d30914841b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 06:04:43.146674 5321 log.cc:1196] T 85d3a2f8d4d944ac94565323c63a0b84 P 65412fe579b64b63b7626d30914841b5: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-0/wal/wals/85d3a2f8d4d944ac94565323c63a0b84
I20260808 06:04:43.146677 5322 ts_tablet_manager.cc:1918] T 85d3a2f8d4d944ac94565323c63a0b84 P 95ab12dd45364e63a84fbbc433f8ccf0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 06:04:43.146731 5322 log.cc:1196] T 85d3a2f8d4d944ac94565323c63a0b84 P 95ab12dd45364e63a84fbbc433f8ccf0: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-2/wal/wals/85d3a2f8d4d944ac94565323c63a0b84
I20260808 06:04:43.146822 5324 ts_tablet_manager.cc:1918] T 85d3a2f8d4d944ac94565323c63a0b84 P 62d68523dc894c159f9dc7abc1eff8d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 06:04:43.146886 5324 log.cc:1196] T 85d3a2f8d4d944ac94565323c63a0b84 P 62d68523dc894c159f9dc7abc1eff8d3: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-1/wal/wals/85d3a2f8d4d944ac94565323c63a0b84
I20260808 06:04:43.146929 5322 ts_tablet_manager.cc:1939] T 85d3a2f8d4d944ac94565323c63a0b84 P 95ab12dd45364e63a84fbbc433f8ccf0: Deleting consensus metadata
I20260808 06:04:43.146927 5321 ts_tablet_manager.cc:1939] T 85d3a2f8d4d944ac94565323c63a0b84 P 65412fe579b64b63b7626d30914841b5: Deleting consensus metadata
I20260808 06:04:43.147101 5324 ts_tablet_manager.cc:1939] T 85d3a2f8d4d944ac94565323c63a0b84 P 62d68523dc894c159f9dc7abc1eff8d3: Deleting consensus metadata
I20260808 06:04:43.147552 4379 catalog_manager.cc:4841] TS 95ab12dd45364e63a84fbbc433f8ccf0 (127.1.151.131:34917): tablet 85d3a2f8d4d944ac94565323c63a0b84 (table test-table [id=4f30c9edcb954d8dbf5458c9bf3974aa]) successfully deleted
I20260808 06:04:43.147588 4380 catalog_manager.cc:4841] TS 65412fe579b64b63b7626d30914841b5 (127.1.151.129:34411): tablet 85d3a2f8d4d944ac94565323c63a0b84 (table test-table [id=4f30c9edcb954d8dbf5458c9bf3974aa]) successfully deleted
I20260808 06:04:43.147760 4380 catalog_manager.cc:4841] TS 62d68523dc894c159f9dc7abc1eff8d3 (127.1.151.130:45285): tablet 85d3a2f8d4d944ac94565323c63a0b84 (table test-table [id=4f30c9edcb954d8dbf5458c9bf3974aa]) successfully deleted
I20260808 06:04:43.147735 4406 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55410:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 06:04:43.148087 4406 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 06:04:43.152104 4608 tablet_service.cc:1467] Processing CreateTablet for tablet fe5581981040404f8056176c27fba687 (DEFAULT_TABLE table=test-table [id=7df957cd993242c28386a1e75a9b0f55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:04:43.152104 4835 tablet_service.cc:1467] Processing CreateTablet for tablet fe5581981040404f8056176c27fba687 (DEFAULT_TABLE table=test-table [id=7df957cd993242c28386a1e75a9b0f55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:04:43.152232 5065 tablet_service.cc:1467] Processing CreateTablet for tablet fe5581981040404f8056176c27fba687 (DEFAULT_TABLE table=test-table [id=7df957cd993242c28386a1e75a9b0f55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:04:43.152359 4835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe5581981040404f8056176c27fba687. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:43.152359 4608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe5581981040404f8056176c27fba687. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:43.152486 5065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe5581981040404f8056176c27fba687. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:43.154109 5338 tablet_bootstrap.cc:492] T fe5581981040404f8056176c27fba687 P 95ab12dd45364e63a84fbbc433f8ccf0: Bootstrap starting.
I20260808 06:04:43.154464 5337 tablet_bootstrap.cc:492] T fe5581981040404f8056176c27fba687 P 62d68523dc894c159f9dc7abc1eff8d3: Bootstrap starting.
I20260808 06:04:43.154552 5339 tablet_bootstrap.cc:492] T fe5581981040404f8056176c27fba687 P 65412fe579b64b63b7626d30914841b5: Bootstrap starting.
I20260808 06:04:43.155180 5337 tablet_bootstrap.cc:654] T fe5581981040404f8056176c27fba687 P 62d68523dc894c159f9dc7abc1eff8d3: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:43.155403 5339 tablet_bootstrap.cc:654] T fe5581981040404f8056176c27fba687 P 65412fe579b64b63b7626d30914841b5: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:43.155529 5338 tablet_bootstrap.cc:654] T fe5581981040404f8056176c27fba687 P 95ab12dd45364e63a84fbbc433f8ccf0: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:43.156154 5337 tablet_bootstrap.cc:492] T fe5581981040404f8056176c27fba687 P 62d68523dc894c159f9dc7abc1eff8d3: No bootstrap required, opened a new log
I20260808 06:04:43.156239 5337 ts_tablet_manager.cc:1397] T fe5581981040404f8056176c27fba687 P 62d68523dc894c159f9dc7abc1eff8d3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 06:04:43.156620 5337 raft_consensus.cc:348] T fe5581981040404f8056176c27fba687 P 62d68523dc894c159f9dc7abc1eff8d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } }
I20260808 06:04:43.156690 5339 tablet_bootstrap.cc:492] T fe5581981040404f8056176c27fba687 P 65412fe579b64b63b7626d30914841b5: No bootstrap required, opened a new log
I20260808 06:04:43.156738 5337 raft_consensus.cc:374] T fe5581981040404f8056176c27fba687 P 62d68523dc894c159f9dc7abc1eff8d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:43.156769 5337 raft_consensus.cc:729] T fe5581981040404f8056176c27fba687 P 62d68523dc894c159f9dc7abc1eff8d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62d68523dc894c159f9dc7abc1eff8d3, State: Initialized, Role: FOLLOWER
I20260808 06:04:43.156777 5339 ts_tablet_manager.cc:1397] T fe5581981040404f8056176c27fba687 P 65412fe579b64b63b7626d30914841b5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 06:04:43.156890 5337 consensus_queue.cc:260] T fe5581981040404f8056176c27fba687 P 62d68523dc894c159f9dc7abc1eff8d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } }
I20260808 06:04:43.157011 5338 tablet_bootstrap.cc:492] T fe5581981040404f8056176c27fba687 P 95ab12dd45364e63a84fbbc433f8ccf0: No bootstrap required, opened a new log
I20260808 06:04:43.157089 5338 ts_tablet_manager.cc:1397] T fe5581981040404f8056176c27fba687 P 95ab12dd45364e63a84fbbc433f8ccf0: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 06:04:43.157160 5337 ts_tablet_manager.cc:1428] T fe5581981040404f8056176c27fba687 P 62d68523dc894c159f9dc7abc1eff8d3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 06:04:43.157152 5339 raft_consensus.cc:348] T fe5581981040404f8056176c27fba687 P 65412fe579b64b63b7626d30914841b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } }
I20260808 06:04:43.157263 5339 raft_consensus.cc:374] T fe5581981040404f8056176c27fba687 P 65412fe579b64b63b7626d30914841b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:43.157305 5339 raft_consensus.cc:729] T fe5581981040404f8056176c27fba687 P 65412fe579b64b63b7626d30914841b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65412fe579b64b63b7626d30914841b5, State: Initialized, Role: FOLLOWER
I20260808 06:04:43.157428 5339 consensus_queue.cc:260] T fe5581981040404f8056176c27fba687 P 65412fe579b64b63b7626d30914841b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } }
I20260808 06:04:43.157528 5338 raft_consensus.cc:348] T fe5581981040404f8056176c27fba687 P 95ab12dd45364e63a84fbbc433f8ccf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } }
I20260808 06:04:43.157677 5338 raft_consensus.cc:374] T fe5581981040404f8056176c27fba687 P 95ab12dd45364e63a84fbbc433f8ccf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:43.157701 5339 ts_tablet_manager.cc:1428] T fe5581981040404f8056176c27fba687 P 65412fe579b64b63b7626d30914841b5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 06:04:43.157740 5338 raft_consensus.cc:729] T fe5581981040404f8056176c27fba687 P 95ab12dd45364e63a84fbbc433f8ccf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95ab12dd45364e63a84fbbc433f8ccf0, State: Initialized, Role: FOLLOWER
I20260808 06:04:43.157864 5338 consensus_queue.cc:260] T fe5581981040404f8056176c27fba687 P 95ab12dd45364e63a84fbbc433f8ccf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } }
I20260808 06:04:43.158107 5338 ts_tablet_manager.cc:1428] T fe5581981040404f8056176c27fba687 P 95ab12dd45364e63a84fbbc433f8ccf0: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260808 06:04:43.446301 5313 raft_consensus.cc:482] T fe5581981040404f8056176c27fba687 P 65412fe579b64b63b7626d30914841b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:04:43.446441 5313 raft_consensus.cc:504] T fe5581981040404f8056176c27fba687 P 65412fe579b64b63b7626d30914841b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } }
I20260808 06:04:43.446933 5313 leader_election.cc:288] T fe5581981040404f8056176c27fba687 P 65412fe579b64b63b7626d30914841b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62d68523dc894c159f9dc7abc1eff8d3 (127.1.151.130:45285), 95ab12dd45364e63a84fbbc433f8ccf0 (127.1.151.131:34917)
I20260808 06:04:43.447376 4863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe5581981040404f8056176c27fba687" candidate_uuid: "65412fe579b64b63b7626d30914841b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62d68523dc894c159f9dc7abc1eff8d3" is_pre_election: true
I20260808 06:04:43.447400 5094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe5581981040404f8056176c27fba687" candidate_uuid: "65412fe579b64b63b7626d30914841b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" is_pre_election: true
I20260808 06:04:43.447535 4863 raft_consensus.cc:2454] T fe5581981040404f8056176c27fba687 P 62d68523dc894c159f9dc7abc1eff8d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65412fe579b64b63b7626d30914841b5 in term 0.
I20260808 06:04:43.447551 5094 raft_consensus.cc:2454] T fe5581981040404f8056176c27fba687 P 95ab12dd45364e63a84fbbc433f8ccf0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65412fe579b64b63b7626d30914841b5 in term 0.
I20260808 06:04:43.447845 4539 leader_election.cc:302] T fe5581981040404f8056176c27fba687 P 65412fe579b64b63b7626d30914841b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62d68523dc894c159f9dc7abc1eff8d3, 65412fe579b64b63b7626d30914841b5; no voters:
I20260808 06:04:43.448017 5313 raft_consensus.cc:2781] T fe5581981040404f8056176c27fba687 P 65412fe579b64b63b7626d30914841b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 06:04:43.448081 5313 raft_consensus.cc:482] T fe5581981040404f8056176c27fba687 P 65412fe579b64b63b7626d30914841b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 06:04:43.448128 5313 raft_consensus.cc:3037] T fe5581981040404f8056176c27fba687 P 65412fe579b64b63b7626d30914841b5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:43.449155 5313 raft_consensus.cc:504] T fe5581981040404f8056176c27fba687 P 65412fe579b64b63b7626d30914841b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } }
I20260808 06:04:43.449482 5313 leader_election.cc:288] T fe5581981040404f8056176c27fba687 P 65412fe579b64b63b7626d30914841b5 [CANDIDATE]: Term 1 election: Requested vote from peers 62d68523dc894c159f9dc7abc1eff8d3 (127.1.151.130:45285), 95ab12dd45364e63a84fbbc433f8ccf0 (127.1.151.131:34917)
I20260808 06:04:43.449819 5094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe5581981040404f8056176c27fba687" candidate_uuid: "65412fe579b64b63b7626d30914841b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95ab12dd45364e63a84fbbc433f8ccf0"
I20260808 06:04:43.449820 4863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe5581981040404f8056176c27fba687" candidate_uuid: "65412fe579b64b63b7626d30914841b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62d68523dc894c159f9dc7abc1eff8d3"
I20260808 06:04:43.449949 4863 raft_consensus.cc:3037] T fe5581981040404f8056176c27fba687 P 62d68523dc894c159f9dc7abc1eff8d3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:43.449949 5094 raft_consensus.cc:3037] T fe5581981040404f8056176c27fba687 P 95ab12dd45364e63a84fbbc433f8ccf0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:43.451133 5094 raft_consensus.cc:2454] T fe5581981040404f8056176c27fba687 P 95ab12dd45364e63a84fbbc433f8ccf0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65412fe579b64b63b7626d30914841b5 in term 1.
I20260808 06:04:43.451133 4863 raft_consensus.cc:2454] T fe5581981040404f8056176c27fba687 P 62d68523dc894c159f9dc7abc1eff8d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65412fe579b64b63b7626d30914841b5 in term 1.
I20260808 06:04:43.451416 4535 leader_election.cc:302] T fe5581981040404f8056176c27fba687 P 65412fe579b64b63b7626d30914841b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65412fe579b64b63b7626d30914841b5, 95ab12dd45364e63a84fbbc433f8ccf0; no voters:
I20260808 06:04:43.451572 5313 raft_consensus.cc:2781] T fe5581981040404f8056176c27fba687 P 65412fe579b64b63b7626d30914841b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:04:43.451696 5313 raft_consensus.cc:686] T fe5581981040404f8056176c27fba687 P 65412fe579b64b63b7626d30914841b5 [term 1 LEADER]: Becoming Leader. State: Replica: 65412fe579b64b63b7626d30914841b5, State: Running, Role: LEADER
I20260808 06:04:43.451840 5313 consensus_queue.cc:237] T fe5581981040404f8056176c27fba687 P 65412fe579b64b63b7626d30914841b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } }
I20260808 06:04:43.453456 4408 catalog_manager.cc:5495] T fe5581981040404f8056176c27fba687 P 65412fe579b64b63b7626d30914841b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 65412fe579b64b63b7626d30914841b5 (127.1.151.129). New cstate: current_term: 1 leader_uuid: "65412fe579b64b63b7626d30914841b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65412fe579b64b63b7626d30914841b5" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 34411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 } health_report { overall_health: UNKNOWN } } }
I20260808 06:04:43.528447 5094 raft_consensus.cc:1261] T fe5581981040404f8056176c27fba687 P 95ab12dd45364e63a84fbbc433f8ccf0 [term 1 FOLLOWER]: Refusing update from remote peer 65412fe579b64b63b7626d30914841b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:04:43.528505 4863 raft_consensus.cc:1261] T fe5581981040404f8056176c27fba687 P 62d68523dc894c159f9dc7abc1eff8d3 [term 1 FOLLOWER]: Refusing update from remote peer 65412fe579b64b63b7626d30914841b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:04:43.528993 5354 consensus_queue.cc:1035] T fe5581981040404f8056176c27fba687 P 65412fe579b64b63b7626d30914841b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62d68523dc894c159f9dc7abc1eff8d3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:04:43.529152 5313 consensus_queue.cc:1035] T fe5581981040404f8056176c27fba687 P 65412fe579b64b63b7626d30914841b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95ab12dd45364e63a84fbbc433f8ccf0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 34917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 06:04:43.542135 5300 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 06:04:45.090270 4403 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:04:45.090420 4403 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:04:45.092830 4403 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 43452437015741b889520f1841311734: Sending DeleteTablet for 3 replicas of tablet fe5581981040404f8056176c27fba687
I20260808 06:04:45.093289 4608 tablet_service.cc:1514] Processing DeleteTablet for tablet fe5581981040404f8056176c27fba687 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:04:45 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:60122
I20260808 06:04:45.093381 4835 tablet_service.cc:1514] Processing DeleteTablet for tablet fe5581981040404f8056176c27fba687 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:04:45 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:50562
I20260808 06:04:45.093394 5065 tablet_service.cc:1514] Processing DeleteTablet for tablet fe5581981040404f8056176c27fba687 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:04:45 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:50340
I20260808 06:04:45.093755 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 4515
I20260808 06:04:45.093752 5405 tablet_replica.cc:330] T fe5581981040404f8056176c27fba687 P 62d68523dc894c159f9dc7abc1eff8d3: stopping tablet replica
I20260808 06:04:45.097855 5405 raft_consensus.cc:2229] T fe5581981040404f8056176c27fba687 P 62d68523dc894c159f9dc7abc1eff8d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:04:45.097994 5405 raft_consensus.cc:2258] T fe5581981040404f8056176c27fba687 P 62d68523dc894c159f9dc7abc1eff8d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:04:45.099848 5405 ts_tablet_manager.cc:1905] T fe5581981040404f8056176c27fba687 P 62d68523dc894c159f9dc7abc1eff8d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:04:45.102013 5405 ts_tablet_manager.cc:1918] T fe5581981040404f8056176c27fba687 P 62d68523dc894c159f9dc7abc1eff8d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 06:04:45.102089 5405 log.cc:1196] T fe5581981040404f8056176c27fba687 P 62d68523dc894c159f9dc7abc1eff8d3: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-1/wal/wals/fe5581981040404f8056176c27fba687
I20260808 06:04:45.102381 5405 ts_tablet_manager.cc:1939] T fe5581981040404f8056176c27fba687 P 62d68523dc894c159f9dc7abc1eff8d3: Deleting consensus metadata
I20260808 06:04:45.102468 5406 tablet_replica.cc:330] T fe5581981040404f8056176c27fba687 P 95ab12dd45364e63a84fbbc433f8ccf0: stopping tablet replica
I20260808 06:04:45.102649 5406 raft_consensus.cc:2229] T fe5581981040404f8056176c27fba687 P 95ab12dd45364e63a84fbbc433f8ccf0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:04:45.102780 5406 raft_consensus.cc:2258] T fe5581981040404f8056176c27fba687 P 95ab12dd45364e63a84fbbc433f8ccf0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:04:45.103382 5406 ts_tablet_manager.cc:1905] T fe5581981040404f8056176c27fba687 P 95ab12dd45364e63a84fbbc433f8ccf0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:04:45.103576 4380 catalog_manager.cc:4841] TS 62d68523dc894c159f9dc7abc1eff8d3 (127.1.151.130:45285): tablet fe5581981040404f8056176c27fba687 (table test-table [id=7df957cd993242c28386a1e75a9b0f55]) successfully deleted
I20260808 06:04:45.105508 5406 ts_tablet_manager.cc:1918] T fe5581981040404f8056176c27fba687 P 95ab12dd45364e63a84fbbc433f8ccf0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 06:04:45.105588 5406 log.cc:1196] T fe5581981040404f8056176c27fba687 P 95ab12dd45364e63a84fbbc433f8ccf0: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786169070948234-1630-0/minicluster-data/ts-2/wal/wals/fe5581981040404f8056176c27fba687
I20260808 06:04:45.105800 5406 ts_tablet_manager.cc:1939] T fe5581981040404f8056176c27fba687 P 95ab12dd45364e63a84fbbc433f8ccf0: Deleting consensus metadata
I20260808 06:04:45.106474 4379 catalog_manager.cc:4841] TS 95ab12dd45364e63a84fbbc433f8ccf0 (127.1.151.131:34917): tablet fe5581981040404f8056176c27fba687 (table test-table [id=7df957cd993242c28386a1e75a9b0f55]) successfully deleted
W20260808 06:04:45.107790 4380 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.151.129:34411 (error 108)
I20260808 06:04:45.108088 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 4731
I20260808 06:04:45.111385 5408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:45.110932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:34411 (local address 127.0.0.1:38000)
0808 06:04:45.111212 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:45.111293 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.129:34411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":92,"threads_started":1}
W20260808 06:04:45.111538 4380 catalog_manager.cc:4584] TS 65412fe579b64b63b7626d30914841b5 (127.1.151.129:34411): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fe5581981040404f8056176c27fba687: Network error: Client connection negotiation failed: client connection to 127.1.151.129:34411: connect: Connection refused (error 111)
I20260808 06:04:45.120390 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 4950
I20260808 06:04:45.137945 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 4357
2026-08-08T06:04:45Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6366 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:04:45 dist-test-slave-zj8q krb5kdc[5411](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:04:45 dist-test-slave-zj8q krb5kdc[5411](info): set up 2 sockets
Aug 08 06:04:45 dist-test-slave-zj8q krb5kdc[5411](info): commencing operation
krb5kdc: starting...
W20260808 06:04:46.314111 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:04:46 dist-test-slave-zj8q krb5kdc[5411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169086, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:04:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:04:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.151.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
I20260808 06:04:46.432154 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:42447
--webserver_interface=127.1.151.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:40247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:42447
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:04:46.541508 5429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:46.541754 5429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:46.541792 5429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:46.545732 5429 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:04:46.545784 5429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:46.545802 5429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:46.545816 5429 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:04:46.545830 5429 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:04:46.550885 5429 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:40247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:42447
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:42447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:46.551836 5429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:46.552807 5429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:46.561561 5438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:46.561548 5439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:46.561683 5429 server_base.cc:1034] running on GCE node
W20260808 06:04:46.561548 5442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:46.562448 5429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:46.563597 5429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:46.564756 5429 hybrid_clock.cc:648] HybridClock initialized: now 1786169086564733 us; error 37 us; skew 500 ppm
Aug 08 06:04:46 dist-test-slave-zj8q krb5kdc[5411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169086, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:04:46.571609 5429 init.cc:377] Logged in from keytab as kudu/127.1.151.190@KRBTEST.COM (short username kudu)
I20260808 06:04:46.572516 5429 webserver.cc:460] Webserver started at http://127.1.151.190:45429/ using document root <none> and password file <none>
I20260808 06:04:46.573041 5429 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:46.573099 5429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:46.573282 5429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:46.575392 5429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "42fc452f91ce498890106cee5e898551"
format_stamp: "Formatted at 2026-08-08 06:04:46 on dist-test-slave-zj8q"
server_key: "2299e1a75d4ba3e28e1b186c5675e7fe"
server_key_iv: "657a35c66bb032907d1af7da07264ebc"
server_key_version: "encryptionkey@0"
I20260808 06:04:46.575909 5429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "42fc452f91ce498890106cee5e898551"
format_stamp: "Formatted at 2026-08-08 06:04:46 on dist-test-slave-zj8q"
server_key: "2299e1a75d4ba3e28e1b186c5675e7fe"
server_key_iv: "657a35c66bb032907d1af7da07264ebc"
server_key_version: "encryptionkey@0"
I20260808 06:04:46.579620 5429 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 06:04:46.581971 5456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:46.583225 5429 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:04:46.583334 5429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "42fc452f91ce498890106cee5e898551"
format_stamp: "Formatted at 2026-08-08 06:04:46 on dist-test-slave-zj8q"
server_key: "2299e1a75d4ba3e28e1b186c5675e7fe"
server_key_iv: "657a35c66bb032907d1af7da07264ebc"
server_key_version: "encryptionkey@0"
I20260808 06:04:46.583421 5429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:46.610986 5429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:46.613799 5429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:46.613991 5429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:46.620867 5429 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:42447
I20260808 06:04:46.620893 5535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:42447 every 8 connection(s)
I20260808 06:04:46.621783 5429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:04:46.624493 5536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:46.628923 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 5429
I20260808 06:04:46.629055 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:04:46.629347 1630 external_mini_cluster.cc:1428] Setting key 08b3cb8d776189c8a43132467c5fcdd4
I20260808 06:04:46.630277 5536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42fc452f91ce498890106cee5e898551: Bootstrap starting.
I20260808 06:04:46.632526 5536 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42fc452f91ce498890106cee5e898551: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:46.633416 5536 log.cc:824] T 00000000000000000000000000000000 P 42fc452f91ce498890106cee5e898551: Log is configured to *not* fsync() on all Append() calls
I20260808 06:04:46.635596 5536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42fc452f91ce498890106cee5e898551: No bootstrap required, opened a new log
Aug 08 06:04:46 dist-test-slave-zj8q krb5kdc[5411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169086, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:04:46.639979 5536 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42fc452f91ce498890106cee5e898551 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42fc452f91ce498890106cee5e898551" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 42447 } }
I20260808 06:04:46.640266 5536 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42fc452f91ce498890106cee5e898551 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:46.640326 5536 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42fc452f91ce498890106cee5e898551 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42fc452f91ce498890106cee5e898551, State: Initialized, Role: FOLLOWER
I20260808 06:04:46.640822 5536 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42fc452f91ce498890106cee5e898551 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42fc452f91ce498890106cee5e898551" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 42447 } }
I20260808 06:04:46.640986 5536 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42fc452f91ce498890106cee5e898551 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:04:46.641054 5536 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42fc452f91ce498890106cee5e898551 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:04:46.641165 5536 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42fc452f91ce498890106cee5e898551 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:46.642309 5536 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42fc452f91ce498890106cee5e898551 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42fc452f91ce498890106cee5e898551" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 42447 } }
I20260808 06:04:46.642663 5536 leader_election.cc:302] T 00000000000000000000000000000000 P 42fc452f91ce498890106cee5e898551 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42fc452f91ce498890106cee5e898551; no voters:
I20260808 06:04:46.642966 5536 leader_election.cc:288] T 00000000000000000000000000000000 P 42fc452f91ce498890106cee5e898551 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:04:46.643071 5547 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42fc452f91ce498890106cee5e898551 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:04:46.643282 5547 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42fc452f91ce498890106cee5e898551 [term 1 LEADER]: Becoming Leader. State: Replica: 42fc452f91ce498890106cee5e898551, State: Running, Role: LEADER
I20260808 06:04:46.644104 5545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:46.630710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36306 (local address 127.1.151.190:42447)
0808 06:04:46.631086 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:46.631101 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:46.631136 (+ 35us) server_negotiation.cc:407] Connection header received
0808 06:04:46.631742 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:46.631759 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:46.632066 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:46.632325 (+ 259us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:46.633261 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:46.634334 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:46.635486 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:46.636008 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:46.636127 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:46.638877 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:46.638884 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:46.638901 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:46.638933 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:46.640772 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:46.641218 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:46.641224 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:46.641231 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:46.641376 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:46.641631 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:46.641634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:46.641635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:46.642194 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:46.642385 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:46.642649 (+ 264us) server_negotiation.cc:299] Negotiation successful
0808 06:04:46.642973 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":94,"threads_started":1}
I20260808 06:04:46.644294 5536 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42fc452f91ce498890106cee5e898551 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:04:46.644424 5547 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42fc452f91ce498890106cee5e898551 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42fc452f91ce498890106cee5e898551" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 42447 } }
I20260808 06:04:46.647089 5549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42fc452f91ce498890106cee5e898551 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42fc452f91ce498890106cee5e898551. Latest consensus state: current_term: 1 leader_uuid: "42fc452f91ce498890106cee5e898551" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42fc452f91ce498890106cee5e898551" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 42447 } } }
I20260808 06:04:46.647217 5549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42fc452f91ce498890106cee5e898551 [sys.catalog]: This master's current role is: LEADER
I20260808 06:04:46.647519 5548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42fc452f91ce498890106cee5e898551 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42fc452f91ce498890106cee5e898551" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42fc452f91ce498890106cee5e898551" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 42447 } } }
I20260808 06:04:46.647615 5548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42fc452f91ce498890106cee5e898551 [sys.catalog]: This master's current role is: LEADER
W20260808 06:04:46.649003 5563 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 42fc452f91ce498890106cee5e898551: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 06:04:46.649071 5563 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 06:04:46.649171 5565 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:04:46.649864 5565 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:04:46.655514 5565 catalog_manager.cc:1347] Generated new cluster ID: 281cf5ca64d44664b1bbe56d64eec364
I20260808 06:04:46.655571 5565 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:04:46.670368 5565 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:04:46.671288 5565 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:04:46.683506 5565 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42fc452f91ce498890106cee5e898551: Generated new TSK 0
I20260808 06:04:46.684424 5565 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.151.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab.
Entry for principal kudu/127.1.151.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab.
I20260808 06:04:46.712917 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.129:0
--local_ip_for_outbound_sockets=127.1.151.129
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:42447
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:40247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:04:46.824616 5580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:46.824855 5580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:46.824894 5580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:46.828825 5580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:46.828876 5580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:46.828928 5580 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.129
I20260808 06:04:46.833887 5580 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:40247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:42447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.129
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:46.834889 5580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:46.835867 5580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:46.845276 5593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:46.845271 5595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:46.845350 5592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:46.845506 5580 server_base.cc:1034] running on GCE node
I20260808 06:04:46.845930 5580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:46.846599 5580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:46.847759 5580 hybrid_clock.cc:648] HybridClock initialized: now 1786169086847743 us; error 28 us; skew 500 ppm
Aug 08 06:04:46 dist-test-slave-zj8q krb5kdc[5411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169086, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:04:46.850448 5580 init.cc:377] Logged in from keytab as kudu/127.1.151.129@KRBTEST.COM (short username kudu)
I20260808 06:04:46.851274 5580 webserver.cc:460] Webserver started at http://127.1.151.129:37389/ using document root <none> and password file <none>
I20260808 06:04:46.851784 5580 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:46.851841 5580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:46.852005 5580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:46.853984 5580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-0/data/instance:
uuid: "abd2a0871d2d4e009e1f8720e948842c"
format_stamp: "Formatted at 2026-08-08 06:04:46 on dist-test-slave-zj8q"
server_key: "a0cfdc98d13f36d47108cedd46720c49"
server_key_iv: "123c06e37647cd751b691959913db3c8"
server_key_version: "encryptionkey@0"
I20260808 06:04:46.854542 5580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance:
uuid: "abd2a0871d2d4e009e1f8720e948842c"
format_stamp: "Formatted at 2026-08-08 06:04:46 on dist-test-slave-zj8q"
server_key: "a0cfdc98d13f36d47108cedd46720c49"
server_key_iv: "123c06e37647cd751b691959913db3c8"
server_key_version: "encryptionkey@0"
I20260808 06:04:46.858193 5580 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 06:04:46.860517 5603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:46.861660 5580 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:04:46.861780 5580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-0/wal
uuid: "abd2a0871d2d4e009e1f8720e948842c"
format_stamp: "Formatted at 2026-08-08 06:04:46 on dist-test-slave-zj8q"
server_key: "a0cfdc98d13f36d47108cedd46720c49"
server_key_iv: "123c06e37647cd751b691959913db3c8"
server_key_version: "encryptionkey@0"
I20260808 06:04:46.861869 5580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:46.890110 5580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:46.892904 5580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:46.893085 5580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:46.893693 5580 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:04:46.894552 5580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:04:46.894603 5580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:46.894656 5580 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:04:46.894686 5580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:46.903155 5580 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.129:39153
I20260808 06:04:46.903185 5798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.129:39153 every 8 connection(s)
I20260808 06:04:46.903985 5580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
I20260808 06:04:46.909711 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 5580
I20260808 06:04:46.909828 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance
I20260808 06:04:46.910070 1630 external_mini_cluster.cc:1428] Setting key 8ae5f6b2fb151cfe5b22e4f76c582663
Aug 08 06:04:46 dist-test-slave-zj8q krb5kdc[5411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169086, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:04:46.917339 5545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:46.905724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:36637 (local address 127.1.151.190:42447)
0808 06:04:46.905882 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:46.905885 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:46.906704 (+ 819us) server_negotiation.cc:407] Connection header received
0808 06:04:46.908012 (+ 1308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:46.908015 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:46.908061 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:46.908135 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:46.909201 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:46.909759 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:46.911011 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:46.911283 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:46.911381 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:46.913895 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:46.913898 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:46.913900 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:46.913923 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:46.915253 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:46.915744 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:46.915748 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:46.915749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:46.915826 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:46.916145 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:46.916148 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:46.916150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:46.916352 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:46.916450 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:46.917091 (+ 641us) server_negotiation.cc:299] Negotiation successful
0808 06:04:46.917216 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 06:04:46.918556 5801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:46.906098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:42447 (local address 127.1.151.129:36637)
0808 06:04:46.906555 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:46.906595 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:46.907807 (+ 1212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:46.908293 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:46.908299 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:46.908627 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:46.909074 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:46.909087 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:46.909889 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:46.909892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:46.910903 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:46.910914 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:46.911440 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:46.911445 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:46.911624 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:46.912171 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:04:46.912188 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:46.913739 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:46.915363 (+ 1624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:46.915370 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:46.915385 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:46.915649 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:46.915909 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:46.915912 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:46.915914 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:46.916055 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:46.916488 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:46.916495 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:46.916827 (+ 332us) client_negotiation.cc:769] Sending connection context
0808 06:04:46.917044 (+ 217us) client_negotiation.cc:240] Negotiation successful
0808 06:04:46.917415 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":113,"threads_started":1}
I20260808 06:04:46.919806 5799 heartbeater.cc:347] Connected to a master server at 127.1.151.190:42447
I20260808 06:04:46.920075 5799 heartbeater.cc:464] Registering TS with master...
I20260808 06:04:46.920676 5799 heartbeater.cc:511] Master 127.1.151.190:42447 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.151.130@KRBTEST.COM; defaulting to no policy
I20260808 06:04:46.922358 5481 ts_manager.cc:194] Registered new tserver with Master: abd2a0871d2d4e009e1f8720e948842c (127.1.151.129:39153)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.130@KRBTEST.COM" created.
I20260808 06:04:46.923648 5481 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.129@KRBTEST.COM'} at 127.1.151.129:36637
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab.
Entry for principal kudu/127.1.151.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab.
I20260808 06:04:46.936571 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.130:0
--local_ip_for_outbound_sockets=127.1.151.130
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:42447
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:40247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:04:47.047397 5804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:47.047672 5804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:47.047737 5804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:47.051729 5804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:47.051781 5804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:47.051867 5804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.130
I20260808 06:04:47.057112 5804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:40247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:42447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.130
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:47.058120 5804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:47.059131 5804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:47.068467 5816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:47.068498 5826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:47.068492 5817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:47.068670 5804 server_base.cc:1034] running on GCE node
I20260808 06:04:47.069083 5804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:47.069659 5804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:47.070828 5804 hybrid_clock.cc:648] HybridClock initialized: now 1786169087070799 us; error 38 us; skew 500 ppm
Aug 08 06:04:47 dist-test-slave-zj8q krb5kdc[5411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169087, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:04:47.073666 5804 init.cc:377] Logged in from keytab as kudu/127.1.151.130@KRBTEST.COM (short username kudu)
I20260808 06:04:47.074533 5804 webserver.cc:460] Webserver started at http://127.1.151.130:46833/ using document root <none> and password file <none>
I20260808 06:04:47.075081 5804 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:47.075142 5804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:47.075415 5804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:47.077571 5804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-1/data/instance:
uuid: "66bcf0ae40074117aa44bf7e8f0a8e30"
format_stamp: "Formatted at 2026-08-08 06:04:47 on dist-test-slave-zj8q"
server_key: "ec4a76509b456163cd7691daf599acbf"
server_key_iv: "a52f42bdfe9c5c566b8931c116f308be"
server_key_version: "encryptionkey@0"
I20260808 06:04:47.078122 5804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance:
uuid: "66bcf0ae40074117aa44bf7e8f0a8e30"
format_stamp: "Formatted at 2026-08-08 06:04:47 on dist-test-slave-zj8q"
server_key: "ec4a76509b456163cd7691daf599acbf"
server_key_iv: "a52f42bdfe9c5c566b8931c116f308be"
server_key_version: "encryptionkey@0"
I20260808 06:04:47.081991 5804 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 06:04:47.084450 5836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:47.085716 5804 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:04:47.085837 5804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-1/wal
uuid: "66bcf0ae40074117aa44bf7e8f0a8e30"
format_stamp: "Formatted at 2026-08-08 06:04:47 on dist-test-slave-zj8q"
server_key: "ec4a76509b456163cd7691daf599acbf"
server_key_iv: "a52f42bdfe9c5c566b8931c116f308be"
server_key_version: "encryptionkey@0"
I20260808 06:04:47.085937 5804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:47.103992 5804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:47.106849 5804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:47.107028 5804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:47.107591 5804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:04:47.108383 5804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:04:47.108429 5804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:47.108471 5804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:04:47.108501 5804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:47.116881 5804 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.130:41317
I20260808 06:04:47.116916 6021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.130:41317 every 8 connection(s)
I20260808 06:04:47.117776 5804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
I20260808 06:04:47.123021 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 5804
I20260808 06:04:47.123139 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance
I20260808 06:04:47.123406 1630 external_mini_cluster.cc:1428] Setting key c6605c7ab16f4b49e75cbbf0dfb38695
Aug 08 06:04:47 dist-test-slave-zj8q krb5kdc[5411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169087, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:04:47.131927 5545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:47.119621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:52947 (local address 127.1.151.190:42447)
0808 06:04:47.119753 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:47.119757 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:47.120624 (+ 867us) server_negotiation.cc:407] Connection header received
0808 06:04:47.121967 (+ 1343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:47.121971 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:47.122032 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:47.122114 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:47.123371 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:47.123933 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:47.125068 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:47.125290 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:47.125378 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:47.128111 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:47.128114 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:47.128117 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:47.128141 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:47.129744 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:47.130334 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:47.130338 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:47.130340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:47.130441 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:47.130757 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:47.130760 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:47.130762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:47.131001 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:47.131107 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:47.131598 (+ 491us) server_negotiation.cc:299] Negotiation successful
0808 06:04:47.131749 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 06:04:47.132615 6027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:47.120009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:42447 (local address 127.1.151.130:52947)
0808 06:04:47.120465 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:47.120508 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:47.121746 (+ 1238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:47.122341 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:47.122348 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:47.122703 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:47.123215 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:47.123236 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:47.124060 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:47.124064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:47.124948 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:47.124958 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:47.125429 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:47.125433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:47.125614 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:47.126201 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:04:47.126219 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:47.127917 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:47.129868 (+ 1951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:47.129874 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:47.129886 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:47.130234 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:47.130525 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:47.130527 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:47.130528 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:47.130665 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:47.131090 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:47.131095 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:47.131350 (+ 255us) client_negotiation.cc:769] Sending connection context
0808 06:04:47.131500 (+ 150us) client_negotiation.cc:240] Negotiation successful
0808 06:04:47.131754 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":106,"threads_started":1}
I20260808 06:04:47.133792 6024 heartbeater.cc:347] Connected to a master server at 127.1.151.190:42447
I20260808 06:04:47.134063 6024 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.1.151.131@KRBTEST.COM; defaulting to no policy
I20260808 06:04:47.134686 6024 heartbeater.cc:511] Master 127.1.151.190:42447 requested a full tablet report, sending...
I20260808 06:04:47.135793 5481 ts_manager.cc:194] Registered new tserver with Master: 66bcf0ae40074117aa44bf7e8f0a8e30 (127.1.151.130:41317)
I20260808 06:04:47.136469 5481 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.130@KRBTEST.COM'} at 127.1.151.130:52947
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab.
Entry for principal kudu/127.1.151.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab.
I20260808 06:04:47.150766 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.131:0
--local_ip_for_outbound_sockets=127.1.151.131
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:42447
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:40247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:04:47.263314 6033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:47.263562 6033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:47.263620 6033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:47.267678 6033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:47.267735 6033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:47.267813 6033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.131
I20260808 06:04:47.272857 6033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:40247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:42447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.131
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:47.273830 6033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:47.274891 6033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:47.284498 6040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:47.284521 6033 server_base.cc:1034] running on GCE node
W20260808 06:04:47.284504 6045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:47.284699 6041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:47.285166 6033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:47.285964 6033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:47.287168 6033 hybrid_clock.cc:648] HybridClock initialized: now 1786169087287135 us; error 51 us; skew 500 ppm
Aug 08 06:04:47 dist-test-slave-zj8q krb5kdc[5411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169087, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:04:47.289897 6033 init.cc:377] Logged in from keytab as kudu/127.1.151.131@KRBTEST.COM (short username kudu)
I20260808 06:04:47.290927 6033 webserver.cc:460] Webserver started at http://127.1.151.131:41499/ using document root <none> and password file <none>
I20260808 06:04:47.291460 6033 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:47.291517 6033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:47.291704 6033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:47.293699 6033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-2/data/instance:
uuid: "443878f327f3401c9226acd6c1d81fe9"
format_stamp: "Formatted at 2026-08-08 06:04:47 on dist-test-slave-zj8q"
server_key: "33c1010d6457368957ef65167e74b43a"
server_key_iv: "d13874a37554521936c68596ef4fa390"
server_key_version: "encryptionkey@0"
I20260808 06:04:47.294258 6033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance:
uuid: "443878f327f3401c9226acd6c1d81fe9"
format_stamp: "Formatted at 2026-08-08 06:04:47 on dist-test-slave-zj8q"
server_key: "33c1010d6457368957ef65167e74b43a"
server_key_iv: "d13874a37554521936c68596ef4fa390"
server_key_version: "encryptionkey@0"
I20260808 06:04:47.297847 6033 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 06:04:47.300232 6056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:47.301397 6033 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:04:47.301507 6033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-2/wal
uuid: "443878f327f3401c9226acd6c1d81fe9"
format_stamp: "Formatted at 2026-08-08 06:04:47 on dist-test-slave-zj8q"
server_key: "33c1010d6457368957ef65167e74b43a"
server_key_iv: "d13874a37554521936c68596ef4fa390"
server_key_version: "encryptionkey@0"
I20260808 06:04:47.301599 6033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:47.315979 6033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:47.318591 6033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:47.318758 6033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:47.319304 6033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:04:47.320080 6033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:04:47.320137 6033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:47.320188 6033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:04:47.320212 6033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:47.328356 6033 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.131:35355
I20260808 06:04:47.328384 6246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.131:35355 every 8 connection(s)
I20260808 06:04:47.329176 6033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
I20260808 06:04:47.336946 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 6033
I20260808 06:04:47.337074 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance
I20260808 06:04:47.337450 1630 external_mini_cluster.cc:1428] Setting key 19eb2b274e7d1ca37dc54f3c545e9e10
Aug 08 06:04:47 dist-test-slave-zj8q krb5kdc[5411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169087, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:04:47.342303 5545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:47.330950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:59329 (local address 127.1.151.190:42447)
0808 06:04:47.331097 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:47.331102 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:47.331931 (+ 829us) server_negotiation.cc:407] Connection header received
0808 06:04:47.333181 (+ 1250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:47.333184 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:47.333232 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:47.333302 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:47.334419 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:47.334939 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:47.336080 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:47.336301 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:47.336378 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:47.339132 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:47.339143 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:47.339146 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:47.339175 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:47.340561 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:47.341019 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:47.341022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:47.341023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:47.341096 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:47.341351 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:47.341353 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:47.341354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:47.341522 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:47.341596 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:47.342054 (+ 458us) server_negotiation.cc:299] Negotiation successful
0808 06:04:47.342178 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 06:04:47.343119 6251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:47.331322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:42447 (local address 127.1.151.131:59329)
0808 06:04:47.331775 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:47.331816 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:47.332974 (+ 1158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:47.333447 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:47.333453 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:47.333783 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:47.334298 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:47.334313 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:47.335078 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:47.335081 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:47.335952 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:47.335962 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:47.336422 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:47.336424 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:47.336578 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:47.337129 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:04:47.337150 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:47.338959 (+ 1809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:47.340673 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:47.340678 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:47.340689 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:47.340938 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:47.341159 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:47.341162 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:47.341163 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:47.341287 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:47.341629 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:47.341634 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:47.341866 (+ 232us) client_negotiation.cc:769] Sending connection context
0808 06:04:47.342018 (+ 152us) client_negotiation.cc:240] Negotiation successful
0808 06:04:47.342319 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":117,"threads_started":1}
I20260808 06:04:47.344091 6247 heartbeater.cc:347] Connected to a master server at 127.1.151.190:42447
I20260808 06:04:47.344337 6247 heartbeater.cc:464] Registering TS with master...
I20260808 06:04:47.344901 6247 heartbeater.cc:511] Master 127.1.151.190:42447 requested a full tablet report, sending...
I20260808 06:04:47.345942 5481 ts_manager.cc:194] Registered new tserver with Master: 443878f327f3401c9226acd6c1d81fe9 (127.1.151.131:35355)
I20260808 06:04:47.346558 5481 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.131@KRBTEST.COM'} at 127.1.151.131:59329
I20260808 06:04:47.351431 1630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 06:04:47.366147 5545 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:04:47.364690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36310 (local address 127.1.151.190:42447)
0808 06:04:47.364803 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:47.364808 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:47.364985 (+ 177us) server_negotiation.cc:407] Connection header received
0808 06:04:47.365145 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:47.365148 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:47.365195 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:47.365259 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:47.366019 (+ 760us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36310: BlockingRecv error: recv got EOF from 127.0.0.1:36310 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 06:04:47.367246 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 5580
I20260808 06:04:47.378712 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 5804
I20260808 06:04:47.396117 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 6033
I20260808 06:04:47.413064 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 5429
2026-08-08T06:04:47Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2276 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:04:47 dist-test-slave-zj8q krb5kdc[6262](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:04:47 dist-test-slave-zj8q krb5kdc[6262](info): set up 2 sockets
Aug 08 06:04:47 dist-test-slave-zj8q krb5kdc[6262](info): commencing operation
krb5kdc: starting...
W20260808 06:04:48.566107 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:04:48 dist-test-slave-zj8q krb5kdc[6262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:04:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:04:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.151.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
I20260808 06:04:48.686579 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:34935
--webserver_interface=127.1.151.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:42995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:34935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:04:48.796399 6284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:48.796643 6284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:48.796680 6284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:48.800619 6284 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:04:48.800691 6284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:48.800709 6284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:48.800724 6284 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:04:48.800737 6284 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:04:48.806136 6284 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:42995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:34935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:34935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:48.807250 6284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:48.808282 6284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:48.816612 6298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:48.816738 6292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:48.816730 6296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:48.816883 6284 server_base.cc:1034] running on GCE node
I20260808 06:04:48.817485 6284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:48.818629 6284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:48.819784 6284 hybrid_clock.cc:648] HybridClock initialized: now 1786169088819771 us; error 24 us; skew 500 ppm
Aug 08 06:04:48 dist-test-slave-zj8q krb5kdc[6262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169088, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:04:48.822484 6284 init.cc:377] Logged in from keytab as kudu/127.1.151.190@KRBTEST.COM (short username kudu)
I20260808 06:04:48.823333 6284 webserver.cc:460] Webserver started at http://127.1.151.190:45093/ using document root <none> and password file <none>
I20260808 06:04:48.823864 6284 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:48.823933 6284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:48.824134 6284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:48.826172 6284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "dc9f97a2be4c4ecf9b46f181f1544dbe"
format_stamp: "Formatted at 2026-08-08 06:04:48 on dist-test-slave-zj8q"
server_key: "1f335b44f1059b967a60419ab25c3cac"
server_key_iv: "41a0d3918d59995f4074a130b1f67222"
server_key_version: "encryptionkey@0"
I20260808 06:04:48.826741 6284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "dc9f97a2be4c4ecf9b46f181f1544dbe"
format_stamp: "Formatted at 2026-08-08 06:04:48 on dist-test-slave-zj8q"
server_key: "1f335b44f1059b967a60419ab25c3cac"
server_key_iv: "41a0d3918d59995f4074a130b1f67222"
server_key_version: "encryptionkey@0"
I20260808 06:04:48.830263 6284 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 06:04:48.832490 6307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:48.833611 6284 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:04:48.833735 6284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "dc9f97a2be4c4ecf9b46f181f1544dbe"
format_stamp: "Formatted at 2026-08-08 06:04:48 on dist-test-slave-zj8q"
server_key: "1f335b44f1059b967a60419ab25c3cac"
server_key_iv: "41a0d3918d59995f4074a130b1f67222"
server_key_version: "encryptionkey@0"
I20260808 06:04:48.833825 6284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:48.852792 6284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:48.855584 6284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:48.855760 6284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:48.862715 6284 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:34935
I20260808 06:04:48.863075 6398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:34935 every 8 connection(s)
I20260808 06:04:48.863955 6284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:04:48.867653 6400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:48.873363 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 6284
I20260808 06:04:48.873492 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:04:48.873853 1630 external_mini_cluster.cc:1428] Setting key 3519716edb2fb1bc504a6bb098761686
I20260808 06:04:48.875492 6400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc9f97a2be4c4ecf9b46f181f1544dbe: Bootstrap starting.
I20260808 06:04:48.877861 6400 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc9f97a2be4c4ecf9b46f181f1544dbe: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:48.878634 6400 log.cc:824] T 00000000000000000000000000000000 P dc9f97a2be4c4ecf9b46f181f1544dbe: Log is configured to *not* fsync() on all Append() calls
I20260808 06:04:48.880751 6400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc9f97a2be4c4ecf9b46f181f1544dbe: No bootstrap required, opened a new log
Aug 08 06:04:48 dist-test-slave-zj8q krb5kdc[6262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:04:48.884887 6400 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc9f97a2be4c4ecf9b46f181f1544dbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9f97a2be4c4ecf9b46f181f1544dbe" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34935 } }
I20260808 06:04:48.885121 6400 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc9f97a2be4c4ecf9b46f181f1544dbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:48.885177 6400 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc9f97a2be4c4ecf9b46f181f1544dbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9f97a2be4c4ecf9b46f181f1544dbe, State: Initialized, Role: FOLLOWER
I20260808 06:04:48.885617 6400 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc9f97a2be4c4ecf9b46f181f1544dbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9f97a2be4c4ecf9b46f181f1544dbe" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34935 } }
I20260808 06:04:48.885782 6400 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc9f97a2be4c4ecf9b46f181f1544dbe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:04:48.885851 6400 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc9f97a2be4c4ecf9b46f181f1544dbe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:04:48.885958 6400 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc9f97a2be4c4ecf9b46f181f1544dbe [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:48.886942 6400 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc9f97a2be4c4ecf9b46f181f1544dbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9f97a2be4c4ecf9b46f181f1544dbe" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34935 } }
I20260808 06:04:48.887317 6400 leader_election.cc:302] T 00000000000000000000000000000000 P dc9f97a2be4c4ecf9b46f181f1544dbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc9f97a2be4c4ecf9b46f181f1544dbe; no voters:
I20260808 06:04:48.887645 6400 leader_election.cc:288] T 00000000000000000000000000000000 P dc9f97a2be4c4ecf9b46f181f1544dbe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:04:48.887761 6407 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc9f97a2be4c4ecf9b46f181f1544dbe [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:04:48.887948 6407 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc9f97a2be4c4ecf9b46f181f1544dbe [term 1 LEADER]: Becoming Leader. State: Replica: dc9f97a2be4c4ecf9b46f181f1544dbe, State: Running, Role: LEADER
I20260808 06:04:48.888253 6407 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc9f97a2be4c4ecf9b46f181f1544dbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9f97a2be4c4ecf9b46f181f1544dbe" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34935 } }
I20260808 06:04:48.888610 6404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:48.875301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44852 (local address 127.1.151.190:34935)
0808 06:04:48.875787 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:48.875805 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:48.875845 (+ 40us) server_negotiation.cc:407] Connection header received
0808 06:04:48.876469 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:48.876485 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:48.876801 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:48.877068 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:48.877999 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:48.879187 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:48.880356 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:48.880848 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:48.881004 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:48.883706 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:48.883713 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:48.883727 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:48.883754 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:48.885413 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:48.885917 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:48.885924 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:48.885931 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:48.886073 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:48.886385 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:48.886387 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:48.886388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:48.886798 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:48.886952 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:48.887282 (+ 330us) server_negotiation.cc:299] Negotiation successful
0808 06:04:48.887625 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":136,"threads_started":1}
I20260808 06:04:48.891052 6400 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc9f97a2be4c4ecf9b46f181f1544dbe [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:04:48.891484 1630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 06:04:48.891969 6410 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc9f97a2be4c4ecf9b46f181f1544dbe [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc9f97a2be4c4ecf9b46f181f1544dbe. Latest consensus state: current_term: 1 leader_uuid: "dc9f97a2be4c4ecf9b46f181f1544dbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9f97a2be4c4ecf9b46f181f1544dbe" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34935 } } }
I20260808 06:04:48.892084 6410 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc9f97a2be4c4ecf9b46f181f1544dbe [sys.catalog]: This master's current role is: LEADER
I20260808 06:04:48.892248 6408 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc9f97a2be4c4ecf9b46f181f1544dbe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc9f97a2be4c4ecf9b46f181f1544dbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9f97a2be4c4ecf9b46f181f1544dbe" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34935 } } }
I20260808 06:04:48.892328 6408 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc9f97a2be4c4ecf9b46f181f1544dbe [sys.catalog]: This master's current role is: LEADER
I20260808 06:04:48.892688 6413 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:04:48.895855 6413 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:04:48.901083 6413 catalog_manager.cc:1347] Generated new cluster ID: a69ecd2097154a47a26a2f07d94f4112
I20260808 06:04:48.901146 6413 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:04:48.916433 6413 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:04:48.917650 6413 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:04:48.923071 6404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:48.917963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44858 (local address 127.1.151.190:34935)
0808 06:04:48.918088 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:48.918093 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:48.918346 (+ 253us) server_negotiation.cc:407] Connection header received
0808 06:04:48.918470 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:48.918473 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:48.918519 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:48.918587 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:48.919123 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:48.919677 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:48.920859 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:48.921116 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:48.921208 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:48.922018 (+ 810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:48.922021 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:48.922023 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:04:48.922532 (+ 509us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:48.922564 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:04:48.922583 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:48.922670 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:48.922853 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 06:04:48.922921 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":54}
I20260808 06:04:48.923965 6413 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc9f97a2be4c4ecf9b46f181f1544dbe: Generated new TSK 0
I20260808 06:04:48.924705 6413 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 06:04:48.933616 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 6284
2026-08-08T06:04:48Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1523 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:04:48 dist-test-slave-zj8q krb5kdc[6456](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:04:48 dist-test-slave-zj8q krb5kdc[6456](info): set up 2 sockets
Aug 08 06:04:48 dist-test-slave-zj8q krb5kdc[6456](info): commencing operation
krb5kdc: starting...
W20260808 06:04:50.084475 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:04:50 dist-test-slave-zj8q krb5kdc[6456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:04:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:04:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.151.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
I20260808 06:04:50.206763 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:41109
--webserver_interface=127.1.151.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:44511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:41109
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:04:50.320039 6481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:50.320294 6481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:50.320333 6481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:50.324399 6481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:04:50.324461 6481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:50.324478 6481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:50.324494 6481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:04:50.324508 6481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:04:50.329751 6481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:44511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:41109
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:41109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:50.330813 6481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:50.331805 6481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:50.339947 6487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:50.339973 6481 server_base.cc:1034] running on GCE node
W20260808 06:04:50.339972 6488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:50.340070 6491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:50.340760 6481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:50.341779 6481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:50.342926 6481 hybrid_clock.cc:648] HybridClock initialized: now 1786169090342899 us; error 38 us; skew 500 ppm
Aug 08 06:04:50 dist-test-slave-zj8q krb5kdc[6456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169090, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:04:50.345459 6481 init.cc:377] Logged in from keytab as kudu/127.1.151.190@KRBTEST.COM (short username kudu)
I20260808 06:04:50.346365 6481 webserver.cc:460] Webserver started at http://127.1.151.190:38187/ using document root <none> and password file <none>
I20260808 06:04:50.346884 6481 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:50.346949 6481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:50.347147 6481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:50.349061 6481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "6609391288ec4c77b2de40478eeb3713"
format_stamp: "Formatted at 2026-08-08 06:04:50 on dist-test-slave-zj8q"
server_key: "bd84f93a434864da890b4ea85f10c430"
server_key_iv: "9d63b7fb04a861dd646ed37b825242a3"
server_key_version: "encryptionkey@0"
I20260808 06:04:50.349579 6481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "6609391288ec4c77b2de40478eeb3713"
format_stamp: "Formatted at 2026-08-08 06:04:50 on dist-test-slave-zj8q"
server_key: "bd84f93a434864da890b4ea85f10c430"
server_key_iv: "9d63b7fb04a861dd646ed37b825242a3"
server_key_version: "encryptionkey@0"
I20260808 06:04:50.353096 6481 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 06:04:50.355387 6505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:50.356565 6481 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 06:04:50.356735 6481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "6609391288ec4c77b2de40478eeb3713"
format_stamp: "Formatted at 2026-08-08 06:04:50 on dist-test-slave-zj8q"
server_key: "bd84f93a434864da890b4ea85f10c430"
server_key_iv: "9d63b7fb04a861dd646ed37b825242a3"
server_key_version: "encryptionkey@0"
I20260808 06:04:50.356838 6481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:50.367731 6481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:50.370841 6481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:50.371021 6481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:50.377698 6481 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:41109
I20260808 06:04:50.377961 6584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:41109 every 8 connection(s)
I20260808 06:04:50.378664 6481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:04:50.381345 6586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:50.383867 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 6481
I20260808 06:04:50.383993 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:04:50.384298 1630 external_mini_cluster.cc:1428] Setting key 97aed31069624ef0a3216482753aee1a
I20260808 06:04:50.387271 6586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6609391288ec4c77b2de40478eeb3713: Bootstrap starting.
I20260808 06:04:50.389712 6586 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6609391288ec4c77b2de40478eeb3713: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:50.390571 6586 log.cc:824] T 00000000000000000000000000000000 P 6609391288ec4c77b2de40478eeb3713: Log is configured to *not* fsync() on all Append() calls
I20260808 06:04:50.392683 6586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6609391288ec4c77b2de40478eeb3713: No bootstrap required, opened a new log
Aug 08 06:04:50 dist-test-slave-zj8q krb5kdc[6456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:04:50.396286 6586 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6609391288ec4c77b2de40478eeb3713 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6609391288ec4c77b2de40478eeb3713" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 41109 } }
I20260808 06:04:50.396513 6586 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6609391288ec4c77b2de40478eeb3713 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:50.396576 6586 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6609391288ec4c77b2de40478eeb3713 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6609391288ec4c77b2de40478eeb3713, State: Initialized, Role: FOLLOWER
I20260808 06:04:50.397004 6586 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6609391288ec4c77b2de40478eeb3713 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6609391288ec4c77b2de40478eeb3713" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 41109 } }
I20260808 06:04:50.397172 6586 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6609391288ec4c77b2de40478eeb3713 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:04:50.397239 6586 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6609391288ec4c77b2de40478eeb3713 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:04:50.397333 6586 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6609391288ec4c77b2de40478eeb3713 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:50.398315 6586 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6609391288ec4c77b2de40478eeb3713 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6609391288ec4c77b2de40478eeb3713" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 41109 } }
I20260808 06:04:50.398620 6586 leader_election.cc:302] T 00000000000000000000000000000000 P 6609391288ec4c77b2de40478eeb3713 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6609391288ec4c77b2de40478eeb3713; no voters:
I20260808 06:04:50.398890 6586 leader_election.cc:288] T 00000000000000000000000000000000 P 6609391288ec4c77b2de40478eeb3713 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:04:50.398988 6597 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6609391288ec4c77b2de40478eeb3713 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:04:50.399083 6592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:50.385631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50876 (local address 127.1.151.190:41109)
0808 06:04:50.386050 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:50.386065 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:50.386099 (+ 34us) server_negotiation.cc:407] Connection header received
0808 06:04:50.386770 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:50.386786 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:50.387095 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:50.387380 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:50.388241 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:50.389214 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:50.390387 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:50.390901 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:50.391072 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:50.393550 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:50.393556 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:50.393571 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:50.393601 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:50.396191 (+ 2590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:50.396631 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:50.396637 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:50.396643 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:50.396766 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:50.397059 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:50.397061 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:50.397062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:50.397509 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:50.397665 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:50.397943 (+ 278us) server_negotiation.cc:299] Negotiation successful
0808 06:04:50.398249 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":111,"threads_started":1}
I20260808 06:04:50.399986 6586 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6609391288ec4c77b2de40478eeb3713 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:04:50.399199 6597 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6609391288ec4c77b2de40478eeb3713 [term 1 LEADER]: Becoming Leader. State: Replica: 6609391288ec4c77b2de40478eeb3713, State: Running, Role: LEADER
I20260808 06:04:50.400365 6597 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6609391288ec4c77b2de40478eeb3713 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6609391288ec4c77b2de40478eeb3713" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 41109 } }
I20260808 06:04:50.402312 6599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6609391288ec4c77b2de40478eeb3713 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6609391288ec4c77b2de40478eeb3713. Latest consensus state: current_term: 1 leader_uuid: "6609391288ec4c77b2de40478eeb3713" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6609391288ec4c77b2de40478eeb3713" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 41109 } } }
I20260808 06:04:50.402416 6599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6609391288ec4c77b2de40478eeb3713 [sys.catalog]: This master's current role is: LEADER
I20260808 06:04:50.402020 6598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6609391288ec4c77b2de40478eeb3713 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6609391288ec4c77b2de40478eeb3713" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6609391288ec4c77b2de40478eeb3713" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 41109 } } }
I20260808 06:04:50.402616 6598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6609391288ec4c77b2de40478eeb3713 [sys.catalog]: This master's current role is: LEADER
W20260808 06:04:50.404640 6619 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6609391288ec4c77b2de40478eeb3713: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 06:04:50.404713 6619 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 06:04:50.404798 6620 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:04:50.405745 6620 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:04:50.411191 6620 catalog_manager.cc:1347] Generated new cluster ID: fa965b719fce4dd58056c4a307903402
I20260808 06:04:50.411242 6620 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:04:50.420640 6620 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:04:50.421482 6620 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:04:50.426561 6620 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6609391288ec4c77b2de40478eeb3713: Generated new TSK 0
I20260808 06:04:50.427219 6620 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.151.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab.
Entry for principal kudu/127.1.151.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab.
I20260808 06:04:50.465304 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.129:0
--local_ip_for_outbound_sockets=127.1.151.129
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:41109
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:44511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:04:50.609248 6628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:50.609504 6628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:50.609545 6628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:50.613555 6628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:50.613607 6628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:50.613669 6628 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.129
I20260808 06:04:50.618781 6628 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:44511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:41109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.129
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:50.619691 6628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:50.620613 6628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:50.629885 6639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:50.629879 6636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:50.629972 6628 server_base.cc:1034] running on GCE node
W20260808 06:04:50.629879 6637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:50.630486 6628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:50.631076 6628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:50.632232 6628 hybrid_clock.cc:648] HybridClock initialized: now 1786169090632217 us; error 26 us; skew 500 ppm
Aug 08 06:04:50 dist-test-slave-zj8q krb5kdc[6456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169090, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:04:50.634886 6628 init.cc:377] Logged in from keytab as kudu/127.1.151.129@KRBTEST.COM (short username kudu)
I20260808 06:04:50.635691 6628 webserver.cc:460] Webserver started at http://127.1.151.129:33481/ using document root <none> and password file <none>
I20260808 06:04:50.636178 6628 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:50.636237 6628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:50.636396 6628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:50.638347 6628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-0/data/instance:
uuid: "2c75560455a34ad283d07f90361e8d55"
format_stamp: "Formatted at 2026-08-08 06:04:50 on dist-test-slave-zj8q"
server_key: "d9349908db4323937bee1472efd5114e"
server_key_iv: "731632387f748b44bfc2a04c162ecf3b"
server_key_version: "encryptionkey@0"
I20260808 06:04:50.638859 6628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance:
uuid: "2c75560455a34ad283d07f90361e8d55"
format_stamp: "Formatted at 2026-08-08 06:04:50 on dist-test-slave-zj8q"
server_key: "d9349908db4323937bee1472efd5114e"
server_key_iv: "731632387f748b44bfc2a04c162ecf3b"
server_key_version: "encryptionkey@0"
I20260808 06:04:50.642491 6628 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 06:04:50.644797 6650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:50.645964 6628 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:04:50.646086 6628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-0/wal
uuid: "2c75560455a34ad283d07f90361e8d55"
format_stamp: "Formatted at 2026-08-08 06:04:50 on dist-test-slave-zj8q"
server_key: "d9349908db4323937bee1472efd5114e"
server_key_iv: "731632387f748b44bfc2a04c162ecf3b"
server_key_version: "encryptionkey@0"
I20260808 06:04:50.646212 6628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:50.657277 6628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:50.660208 6628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:50.660372 6628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:50.660916 6628 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:04:50.661698 6628 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:04:50.661746 6628 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:50.661794 6628 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:04:50.661821 6628 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:50.670224 6628 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.129:43851
I20260808 06:04:50.670259 6823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.129:43851 every 8 connection(s)
I20260808 06:04:50.671026 6628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
I20260808 06:04:50.672526 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 6628
I20260808 06:04:50.672655 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance
I20260808 06:04:50.672972 1630 external_mini_cluster.cc:1428] Setting key f31eb322f16909b951c43e58c5ff3b64
Aug 08 06:04:50 dist-test-slave-zj8q krb5kdc[6456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169090, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.151.130@KRBTEST.COM; defaulting to no policy
I20260808 06:04:50.684839 6592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:50.672728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:39843 (local address 127.1.151.190:41109)
0808 06:04:50.672846 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:50.672851 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:50.673841 (+ 990us) server_negotiation.cc:407] Connection header received
0808 06:04:50.675288 (+ 1447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:50.675292 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:50.675348 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:50.675420 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:50.676723 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:50.677281 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:50.678574 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:50.678831 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:50.678927 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:50.681351 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:50.681355 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:50.681357 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:50.681382 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:50.682778 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:50.683328 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:50.683331 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:50.683333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:50.683415 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:50.683753 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:50.683756 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:50.683758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:50.683961 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:50.684056 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:50.684578 (+ 522us) server_negotiation.cc:299] Negotiation successful
0808 06:04:50.684709 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.130@KRBTEST.COM" created.
I20260808 06:04:50.685945 6827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:50.673206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:41109 (local address 127.1.151.129:39843)
0808 06:04:50.673696 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:50.673746 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:50.675073 (+ 1327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:50.675599 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:50.675606 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:50.675934 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:50.676552 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:50.676570 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:50.677417 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:50.677420 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:50.678453 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:50.678464 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:50.678977 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:50.678982 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:50.679152 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:50.679744 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:04:50.679760 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:50.681180 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:50.682897 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:50.682905 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:50.682922 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:50.683219 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:50.683502 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:50.683505 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:50.683507 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:50.683659 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:50.684086 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:50.684092 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:50.684357 (+ 265us) client_negotiation.cc:769] Sending connection context
0808 06:04:50.684536 (+ 179us) client_negotiation.cc:240] Negotiation successful
0808 06:04:50.684826 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":125,"threads_started":1}
I20260808 06:04:50.687400 6825 heartbeater.cc:347] Connected to a master server at 127.1.151.190:41109
I20260808 06:04:50.687696 6825 heartbeater.cc:464] Registering TS with master...
I20260808 06:04:50.688294 6825 heartbeater.cc:511] Master 127.1.151.190:41109 requested a full tablet report, sending...
I20260808 06:04:50.690021 6534 ts_manager.cc:194] Registered new tserver with Master: 2c75560455a34ad283d07f90361e8d55 (127.1.151.129:43851)
I20260808 06:04:50.691725 6534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.129@KRBTEST.COM'} at 127.1.151.129:39843
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab.
Entry for principal kudu/127.1.151.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab.
I20260808 06:04:50.702366 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.130:0
--local_ip_for_outbound_sockets=127.1.151.130
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:41109
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:44511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:04:50.811977 6832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:50.812263 6832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:50.812309 6832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:50.816202 6832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:50.816255 6832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:50.816310 6832 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.130
I20260808 06:04:50.821537 6832 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:44511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:41109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.130
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:50.822559 6832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:50.823547 6832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 06:04:50.833106 6832 server_base.cc:1034] running on GCE node
W20260808 06:04:50.833011 6838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:50.833022 6842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:50.833014 6845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:50.833663 6832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:50.834311 6832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:50.835471 6832 hybrid_clock.cc:648] HybridClock initialized: now 1786169090835457 us; error 23 us; skew 500 ppm
Aug 08 06:04:50 dist-test-slave-zj8q krb5kdc[6456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169090, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:04:50.838133 6832 init.cc:377] Logged in from keytab as kudu/127.1.151.130@KRBTEST.COM (short username kudu)
I20260808 06:04:50.838976 6832 webserver.cc:460] Webserver started at http://127.1.151.130:42567/ using document root <none> and password file <none>
I20260808 06:04:50.839490 6832 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:50.839547 6832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:50.839708 6832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:50.841687 6832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-1/data/instance:
uuid: "3eed4ec7bc0846e3a80ba98d204b04ce"
format_stamp: "Formatted at 2026-08-08 06:04:50 on dist-test-slave-zj8q"
server_key: "4884f383f43d02316041224354b50b34"
server_key_iv: "aef46ad1a9378d2b3537ba416d53b536"
server_key_version: "encryptionkey@0"
I20260808 06:04:50.842267 6832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance:
uuid: "3eed4ec7bc0846e3a80ba98d204b04ce"
format_stamp: "Formatted at 2026-08-08 06:04:50 on dist-test-slave-zj8q"
server_key: "4884f383f43d02316041224354b50b34"
server_key_iv: "aef46ad1a9378d2b3537ba416d53b536"
server_key_version: "encryptionkey@0"
I20260808 06:04:50.846208 6832 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 06:04:50.848685 6857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:50.849960 6832 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:04:50.850078 6832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-1/wal
uuid: "3eed4ec7bc0846e3a80ba98d204b04ce"
format_stamp: "Formatted at 2026-08-08 06:04:50 on dist-test-slave-zj8q"
server_key: "4884f383f43d02316041224354b50b34"
server_key_iv: "aef46ad1a9378d2b3537ba416d53b536"
server_key_version: "encryptionkey@0"
I20260808 06:04:50.850229 6832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:50.864718 6832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:50.867511 6832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:50.867657 6832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:50.868175 6832 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:04:50.868968 6832 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:04:50.869022 6832 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:50.869059 6832 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:04:50.869076 6832 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:50.877331 6832 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.130:44687
I20260808 06:04:50.877349 7038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.130:44687 every 8 connection(s)
I20260808 06:04:50.878187 6832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
I20260808 06:04:50.878700 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 6832
I20260808 06:04:50.878818 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance
I20260808 06:04:50.879104 1630 external_mini_cluster.cc:1428] Setting key 62aed9a9de17281b4a6b08697e9f211e
Aug 08 06:04:50 dist-test-slave-zj8q krb5kdc[6456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169090, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.151.131@KRBTEST.COM; defaulting to no policy
I20260808 06:04:50.893052 6592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:50.879891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:37895 (local address 127.1.151.190:41109)
0808 06:04:50.880523 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:50.880528 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:50.881084 (+ 556us) server_negotiation.cc:407] Connection header received
0808 06:04:50.882555 (+ 1471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:50.882560 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:50.882627 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:50.882713 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:50.884264 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:50.884807 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:50.886517 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:50.886746 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:50.886826 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:50.889625 (+ 2799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:50.889628 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:50.889630 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:50.889656 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:50.890977 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:50.891610 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:50.891613 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:50.891615 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:50.891701 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:50.892009 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:50.892012 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:50.892013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:50.892203 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:50.892282 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:50.892749 (+ 467us) server_negotiation.cc:299] Negotiation successful
0808 06:04:50.892858 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.131@KRBTEST.COM" created.
I20260808 06:04:50.893930 7045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:50.880413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:41109 (local address 127.1.151.130:37895)
0808 06:04:50.880920 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:50.880972 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:50.882326 (+ 1354us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:50.882916 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:50.882926 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:50.883423 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:50.884102 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:50.884122 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:50.884936 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:50.884939 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:50.886399 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:50.886411 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:50.886902 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:50.886907 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:50.887113 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:50.887705 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:04:50.887727 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:50.889448 (+ 1721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:50.891111 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:50.891116 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:50.891129 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:50.891419 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:50.891785 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:50.891787 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:50.891788 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:50.891924 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:50.892286 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:50.892291 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:50.892527 (+ 236us) client_negotiation.cc:769] Sending connection context
0808 06:04:50.892690 (+ 163us) client_negotiation.cc:240] Negotiation successful
0808 06:04:50.892944 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":134,"threads_started":1}
I20260808 06:04:50.896368 7040 heartbeater.cc:347] Connected to a master server at 127.1.151.190:41109
I20260808 06:04:50.896613 7040 heartbeater.cc:464] Registering TS with master...
I20260808 06:04:50.897121 7040 heartbeater.cc:511] Master 127.1.151.190:41109 requested a full tablet report, sending...
I20260808 06:04:50.898136 6534 ts_manager.cc:194] Registered new tserver with Master: 3eed4ec7bc0846e3a80ba98d204b04ce (127.1.151.130:44687)
I20260808 06:04:50.898798 6534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.130@KRBTEST.COM'} at 127.1.151.130:37895
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab.
Entry for principal kudu/127.1.151.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab.
I20260808 06:04:50.910358 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.131:0
--local_ip_for_outbound_sockets=127.1.151.131
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:41109
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:44511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:04:51.022027 7049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:51.022293 7049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:51.022336 7049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:51.026221 7049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:51.026269 7049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:51.026324 7049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.131
I20260808 06:04:51.031414 7049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:44511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:41109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.131
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:51.032348 7049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:51.033294 7049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 06:04:51.042649 7049 server_base.cc:1034] running on GCE node
W20260808 06:04:51.042593 7055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:51.042585 7061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:51.042599 7054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:51.043290 7049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:51.043934 7049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:51.045116 7049 hybrid_clock.cc:648] HybridClock initialized: now 1786169091045087 us; error 31 us; skew 500 ppm
Aug 08 06:04:51 dist-test-slave-zj8q krb5kdc[6456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169091, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:04:51.047812 7049 init.cc:377] Logged in from keytab as kudu/127.1.151.131@KRBTEST.COM (short username kudu)
I20260808 06:04:51.048712 7049 webserver.cc:460] Webserver started at http://127.1.151.131:39453/ using document root <none> and password file <none>
I20260808 06:04:51.049296 7049 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:51.049361 7049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:51.049561 7049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:51.051666 7049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-2/data/instance:
uuid: "564e89730b2146ae98ca7b8346b42c41"
format_stamp: "Formatted at 2026-08-08 06:04:51 on dist-test-slave-zj8q"
server_key: "8bce1b87c8e2b914de5c250ece5a1786"
server_key_iv: "06ac7f0a5d976759451df4ce5a1d0d3a"
server_key_version: "encryptionkey@0"
I20260808 06:04:51.052198 7049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance:
uuid: "564e89730b2146ae98ca7b8346b42c41"
format_stamp: "Formatted at 2026-08-08 06:04:51 on dist-test-slave-zj8q"
server_key: "8bce1b87c8e2b914de5c250ece5a1786"
server_key_iv: "06ac7f0a5d976759451df4ce5a1d0d3a"
server_key_version: "encryptionkey@0"
I20260808 06:04:51.055871 7049 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260808 06:04:51.058179 7079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:51.059305 7049 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 06:04:51.059439 7049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-2/wal
uuid: "564e89730b2146ae98ca7b8346b42c41"
format_stamp: "Formatted at 2026-08-08 06:04:51 on dist-test-slave-zj8q"
server_key: "8bce1b87c8e2b914de5c250ece5a1786"
server_key_iv: "06ac7f0a5d976759451df4ce5a1d0d3a"
server_key_version: "encryptionkey@0"
I20260808 06:04:51.059528 7049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:51.068223 7049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:51.070683 7049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:51.070842 7049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:51.071365 7049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:04:51.072113 7049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:04:51.072160 7049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:51.072209 7049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:04:51.072238 7049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:51.080245 7049 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.131:35033
I20260808 06:04:51.080272 7240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.131:35033 every 8 connection(s)
I20260808 06:04:51.081033 7049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
I20260808 06:04:51.087073 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 7049
I20260808 06:04:51.087209 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance
I20260808 06:04:51.087499 1630 external_mini_cluster.cc:1428] Setting key a1e431ade2c8933ef4760f24e4703dac
Aug 08 06:04:51 dist-test-slave-zj8q krb5kdc[6456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169091, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:04:51.093828 6592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:51.082825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:48193 (local address 127.1.151.190:41109)
0808 06:04:51.082955 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:51.082958 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:51.083749 (+ 791us) server_negotiation.cc:407] Connection header received
0808 06:04:51.085090 (+ 1341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:51.085093 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:51.085152 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:51.085224 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:51.086414 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:51.087021 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:51.088210 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:51.088430 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:51.088519 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:51.090955 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:51.090958 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:51.090960 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:51.090985 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:51.092160 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:51.092598 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:51.092601 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:51.092603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:51.092682 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:51.092948 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:51.092951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:51.092952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:51.093136 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:51.093210 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:51.093615 (+ 405us) server_negotiation.cc:299] Negotiation successful
0808 06:04:51.093712 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 06:04:51.094655 7244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:51.083116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:41109 (local address 127.1.151.131:48193)
0808 06:04:51.083561 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:51.083635 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:51.084898 (+ 1263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:51.085368 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:51.085375 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:51.085779 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:51.086295 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:51.086311 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:51.087136 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:51.087140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:51.088085 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:51.088095 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:51.088566 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:51.088570 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:51.088772 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:51.089302 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:04:51.089316 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:51.090799 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:51.092266 (+ 1467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:51.092271 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:51.092282 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:51.092523 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:51.092749 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:51.092751 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:51.092753 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:51.092875 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:51.093209 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:51.093214 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:51.093435 (+ 221us) client_negotiation.cc:769] Sending connection context
0808 06:04:51.093580 (+ 145us) client_negotiation.cc:240] Negotiation successful
0808 06:04:51.093828 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":114,"threads_started":1}
I20260808 06:04:51.095631 7241 heartbeater.cc:347] Connected to a master server at 127.1.151.190:41109
I20260808 06:04:51.095840 7241 heartbeater.cc:464] Registering TS with master...
I20260808 06:04:51.096276 7241 heartbeater.cc:511] Master 127.1.151.190:41109 requested a full tablet report, sending...
I20260808 06:04:51.097223 6534 ts_manager.cc:194] Registered new tserver with Master: 564e89730b2146ae98ca7b8346b42c41 (127.1.151.131:35033)
I20260808 06:04:51.097751 6534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.131@KRBTEST.COM'} at 127.1.151.131:48193
I20260808 06:04:51.101086 1630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 06:04:51 dist-test-slave-zj8q krb5kdc[6456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169091, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 06:04:51 dist-test-slave-zj8q krb5kdc[6456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169091, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:04:51.126428 6592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:51.118574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50884 (local address 127.1.151.190:41109)
0808 06:04:51.118712 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:51.118717 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:51.118852 (+ 135us) server_negotiation.cc:407] Connection header received
0808 06:04:51.118973 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:51.118975 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:51.119017 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:51.119074 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:51.119702 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:51.120203 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:51.120966 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:51.121164 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:51.121227 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:51.123477 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:51.123480 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:51.123482 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:51.123507 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:51.124684 (+ 1177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:51.125054 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:51.125056 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:51.125057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:51.125122 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:51.125392 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:51.125394 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:51.125395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:51.125651 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:51.125767 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:51.126020 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 06:04:51.126123 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 06:04:51.126866 6534 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:50884
I20260808 06:04:51.127727 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 6628
I20260808 06:04:51.148914 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 6832
I20260808 06:04:51.156883 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 7049
I20260808 06:04:51.167423 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 6481
2026-08-08T06:04:51Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2227 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:04:51 dist-test-slave-zj8q krb5kdc[7261](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:04:51 dist-test-slave-zj8q krb5kdc[7261](info): set up 2 sockets
Aug 08 06:04:51 dist-test-slave-zj8q krb5kdc[7261](info): commencing operation
krb5kdc: starting...
W20260808 06:04:52.314435 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:04:52 dist-test-slave-zj8q krb5kdc[7261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:04:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:04:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.151.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
I20260808 06:04:52.434862 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:43817
--webserver_interface=127.1.151.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:35765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:43817
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:04:52.544314 7282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:52.544569 7282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:52.544621 7282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:52.548544 7282 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:04:52.548597 7282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:52.548623 7282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:52.548651 7282 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:04:52.548679 7282 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:04:52.553772 7282 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:35765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:43817
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:43817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:52.554818 7282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:52.555806 7282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:52.563969 7290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:52.563956 7291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:52.564023 7293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:52.564174 7282 server_base.cc:1034] running on GCE node
I20260808 06:04:52.564841 7282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:52.566043 7282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:52.567232 7282 hybrid_clock.cc:648] HybridClock initialized: now 1786169092567216 us; error 29 us; skew 500 ppm
Aug 08 06:04:52 dist-test-slave-zj8q krb5kdc[7261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169092, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:04:52.573165 7282 init.cc:377] Logged in from keytab as kudu/127.1.151.190@KRBTEST.COM (short username kudu)
I20260808 06:04:52.574185 7282 webserver.cc:460] Webserver started at http://127.1.151.190:35465/ using document root <none> and password file <none>
I20260808 06:04:52.574774 7282 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:52.574843 7282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:52.575052 7282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:52.577133 7282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "03e8681b918a4555a7f79e9c5a4ee0ca"
format_stamp: "Formatted at 2026-08-08 06:04:52 on dist-test-slave-zj8q"
server_key: "ef876e0ca28be7bc0df02dcbb2f84125"
server_key_iv: "bd120a6d66e6a46622822da8f19e2cb2"
server_key_version: "encryptionkey@0"
I20260808 06:04:52.577673 7282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "03e8681b918a4555a7f79e9c5a4ee0ca"
format_stamp: "Formatted at 2026-08-08 06:04:52 on dist-test-slave-zj8q"
server_key: "ef876e0ca28be7bc0df02dcbb2f84125"
server_key_iv: "bd120a6d66e6a46622822da8f19e2cb2"
server_key_version: "encryptionkey@0"
I20260808 06:04:52.581477 7282 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 06:04:52.583838 7302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:52.585018 7282 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 06:04:52.585140 7282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "03e8681b918a4555a7f79e9c5a4ee0ca"
format_stamp: "Formatted at 2026-08-08 06:04:52 on dist-test-slave-zj8q"
server_key: "ef876e0ca28be7bc0df02dcbb2f84125"
server_key_iv: "bd120a6d66e6a46622822da8f19e2cb2"
server_key_version: "encryptionkey@0"
I20260808 06:04:52.585238 7282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:52.600399 7282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:52.603364 7282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:52.603545 7282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:52.610610 7282 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:43817
I20260808 06:04:52.610708 7377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:43817 every 8 connection(s)
I20260808 06:04:52.611614 7282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:04:52.614372 7378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:52.619861 7378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03e8681b918a4555a7f79e9c5a4ee0ca: Bootstrap starting.
I20260808 06:04:52.620822 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 7282
I20260808 06:04:52.620939 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:04:52.621182 1630 external_mini_cluster.cc:1428] Setting key c5ad442688a1cd9627da07e198d26b0f
I20260808 06:04:52.622584 7378 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03e8681b918a4555a7f79e9c5a4ee0ca: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:52.623461 7378 log.cc:824] T 00000000000000000000000000000000 P 03e8681b918a4555a7f79e9c5a4ee0ca: Log is configured to *not* fsync() on all Append() calls
I20260808 06:04:52.625633 7378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03e8681b918a4555a7f79e9c5a4ee0ca: No bootstrap required, opened a new log
I20260808 06:04:52.629606 7378 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03e8681b918a4555a7f79e9c5a4ee0ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03e8681b918a4555a7f79e9c5a4ee0ca" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 43817 } }
I20260808 06:04:52.629895 7378 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03e8681b918a4555a7f79e9c5a4ee0ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:52.629952 7378 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03e8681b918a4555a7f79e9c5a4ee0ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03e8681b918a4555a7f79e9c5a4ee0ca, State: Initialized, Role: FOLLOWER
Aug 08 06:04:52 dist-test-slave-zj8q krb5kdc[7261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:04:52.630436 7378 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03e8681b918a4555a7f79e9c5a4ee0ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03e8681b918a4555a7f79e9c5a4ee0ca" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 43817 } }
I20260808 06:04:52.630570 7378 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03e8681b918a4555a7f79e9c5a4ee0ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:04:52.630630 7378 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03e8681b918a4555a7f79e9c5a4ee0ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:04:52.630733 7378 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03e8681b918a4555a7f79e9c5a4ee0ca [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:52.631737 7378 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03e8681b918a4555a7f79e9c5a4ee0ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03e8681b918a4555a7f79e9c5a4ee0ca" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 43817 } }
I20260808 06:04:52.632091 7378 leader_election.cc:302] T 00000000000000000000000000000000 P 03e8681b918a4555a7f79e9c5a4ee0ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03e8681b918a4555a7f79e9c5a4ee0ca; no voters:
I20260808 06:04:52.632434 7378 leader_election.cc:288] T 00000000000000000000000000000000 P 03e8681b918a4555a7f79e9c5a4ee0ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:04:52.632524 7390 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03e8681b918a4555a7f79e9c5a4ee0ca [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:04:52.632750 7390 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03e8681b918a4555a7f79e9c5a4ee0ca [term 1 LEADER]: Becoming Leader. State: Replica: 03e8681b918a4555a7f79e9c5a4ee0ca, State: Running, Role: LEADER
I20260808 06:04:52.633117 7390 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03e8681b918a4555a7f79e9c5a4ee0ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03e8681b918a4555a7f79e9c5a4ee0ca" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 43817 } }
I20260808 06:04:52.633667 7378 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03e8681b918a4555a7f79e9c5a4ee0ca [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:04:52.634290 7390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03e8681b918a4555a7f79e9c5a4ee0ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03e8681b918a4555a7f79e9c5a4ee0ca. Latest consensus state: current_term: 1 leader_uuid: "03e8681b918a4555a7f79e9c5a4ee0ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03e8681b918a4555a7f79e9c5a4ee0ca" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 43817 } } }
I20260808 06:04:52.634418 7390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03e8681b918a4555a7f79e9c5a4ee0ca [sys.catalog]: This master's current role is: LEADER
I20260808 06:04:52.634706 7391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03e8681b918a4555a7f79e9c5a4ee0ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03e8681b918a4555a7f79e9c5a4ee0ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03e8681b918a4555a7f79e9c5a4ee0ca" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 43817 } } }
I20260808 06:04:52.634791 7391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03e8681b918a4555a7f79e9c5a4ee0ca [sys.catalog]: This master's current role is: LEADER
I20260808 06:04:52.636858 7384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:52.622370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60704 (local address 127.1.151.190:43817)
0808 06:04:52.622835 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:52.622855 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:52.622900 (+ 45us) server_negotiation.cc:407] Connection header received
0808 06:04:52.623686 (+ 786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:52.623711 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:52.624079 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:52.624409 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:52.625104 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:52.626611 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:52.627465 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:52.628184 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:52.628354 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:52.630915 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:52.630921 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:52.630937 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:52.630968 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:52.632910 (+ 1942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:52.633421 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:52.633426 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:52.633432 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:52.633573 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:52.633895 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:52.633898 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:52.633900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:52.634614 (+ 714us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:52.635540 (+ 926us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:52.635698 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 06:04:52.635994 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":124,"threads_started":1}
W20260808 06:04:52.639061 7410 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 03e8681b918a4555a7f79e9c5a4ee0ca: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 06:04:52.639129 7410 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 06:04:52.639204 7397 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:04:52.639858 7397 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:04:52.644788 7397 catalog_manager.cc:1347] Generated new cluster ID: 324704fbd8444735b15b19a9fd1c4cdb
I20260808 06:04:52.644853 7397 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:04:52.661832 7397 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:04:52.663268 7397 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:04:52.669858 7397 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03e8681b918a4555a7f79e9c5a4ee0ca: Generated new TSK 0
I20260808 06:04:52.670514 7397 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.151.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab.
Entry for principal kudu/127.1.151.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab.
I20260808 06:04:52.704311 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.129:0
--local_ip_for_outbound_sockets=127.1.151.129
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:43817
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:35765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:04:52.817687 7417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:52.817924 7417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:52.817970 7417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:52.821962 7417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:52.822016 7417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:52.822072 7417 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.129
I20260808 06:04:52.827293 7417 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:35765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:43817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.129
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:52.828227 7417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:52.829159 7417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:52.838467 7437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:52.838477 7426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:52.838569 7417 server_base.cc:1034] running on GCE node
W20260808 06:04:52.838465 7430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:52.839129 7417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:52.839751 7417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:52.840914 7417 hybrid_clock.cc:648] HybridClock initialized: now 1786169092840889 us; error 36 us; skew 500 ppm
Aug 08 06:04:52 dist-test-slave-zj8q krb5kdc[7261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169092, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:04:52.843691 7417 init.cc:377] Logged in from keytab as kudu/127.1.151.129@KRBTEST.COM (short username kudu)
I20260808 06:04:52.844511 7417 webserver.cc:460] Webserver started at http://127.1.151.129:33475/ using document root <none> and password file <none>
I20260808 06:04:52.845073 7417 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:52.845144 7417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:52.845350 7417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:52.847545 7417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-0/data/instance:
uuid: "c8fd6a2562db428abe9164ada8fd115b"
format_stamp: "Formatted at 2026-08-08 06:04:52 on dist-test-slave-zj8q"
server_key: "6b29145c497738769e49ee36eb9bea97"
server_key_iv: "5395e8d5a12b12c2144ac5fdfab938a2"
server_key_version: "encryptionkey@0"
I20260808 06:04:52.848089 7417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance:
uuid: "c8fd6a2562db428abe9164ada8fd115b"
format_stamp: "Formatted at 2026-08-08 06:04:52 on dist-test-slave-zj8q"
server_key: "6b29145c497738769e49ee36eb9bea97"
server_key_iv: "5395e8d5a12b12c2144ac5fdfab938a2"
server_key_version: "encryptionkey@0"
I20260808 06:04:52.851878 7417 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260808 06:04:52.854362 7450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:52.855690 7417 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 06:04:52.855804 7417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-0/wal
uuid: "c8fd6a2562db428abe9164ada8fd115b"
format_stamp: "Formatted at 2026-08-08 06:04:52 on dist-test-slave-zj8q"
server_key: "6b29145c497738769e49ee36eb9bea97"
server_key_iv: "5395e8d5a12b12c2144ac5fdfab938a2"
server_key_version: "encryptionkey@0"
I20260808 06:04:52.855898 7417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:52.878739 7417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:52.881673 7417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:52.881843 7417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:52.882449 7417 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:04:52.883281 7417 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:04:52.883330 7417 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:52.883384 7417 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:04:52.883409 7417 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:52.891868 7417 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.129:35477
I20260808 06:04:52.891897 7634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.129:35477 every 8 connection(s)
I20260808 06:04:52.892755 7417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
I20260808 06:04:52.901134 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 7417
I20260808 06:04:52.901253 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance
I20260808 06:04:52.901551 1630 external_mini_cluster.cc:1428] Setting key 41033e76635d125cb463c41cc1b1c0bd
Aug 08 06:04:52 dist-test-slave-zj8q krb5kdc[7261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169092, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:04:52.906322 7384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:52.894607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:33843 (local address 127.1.151.190:43817)
0808 06:04:52.894763 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:52.894767 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:52.895661 (+ 894us) server_negotiation.cc:407] Connection header received
0808 06:04:52.897001 (+ 1340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:52.897004 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:52.897059 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:52.897133 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:52.898269 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:52.898810 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:52.900226 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:52.900462 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:52.900552 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:52.902904 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:52.902908 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:52.902911 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:52.902938 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:52.904227 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:52.904727 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:52.904730 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:52.904732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:52.904830 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:52.905179 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:52.905181 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:52.905183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:52.905406 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:52.905502 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:52.905989 (+ 487us) server_negotiation.cc:299] Negotiation successful
0808 06:04:52.906137 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 06:04:52.907001 7638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:52.895045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:43817 (local address 127.1.151.129:33843)
0808 06:04:52.895494 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:52.895536 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:52.896797 (+ 1261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:52.897280 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:52.897286 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:52.897619 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:52.898102 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:52.898117 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:52.898931 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:52.898935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:52.900100 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:52.900111 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:52.900600 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:52.900604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:52.900743 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:52.901297 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:04:52.901317 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:52.902762 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:52.904342 (+ 1580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:52.904348 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:52.904362 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:52.904619 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:52.904944 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:52.904946 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:52.904948 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:52.905093 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:52.905503 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:52.905508 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:52.905767 (+ 259us) client_negotiation.cc:769] Sending connection context
0808 06:04:52.905917 (+ 150us) client_negotiation.cc:240] Negotiation successful
0808 06:04:52.906213 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":110,"threads_started":1}
I20260808 06:04:52.908209 7636 heartbeater.cc:347] Connected to a master server at 127.1.151.190:43817
I20260808 06:04:52.908420 7636 heartbeater.cc:464] Registering TS with master...
I20260808 06:04:52.908881 7636 heartbeater.cc:511] Master 127.1.151.190:43817 requested a full tablet report, sending...
I20260808 06:04:52.910367 7319 ts_manager.cc:194] Registered new tserver with Master: c8fd6a2562db428abe9164ada8fd115b (127.1.151.129:35477)
I20260808 06:04:52.911453 7319 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.129@KRBTEST.COM'} at 127.1.151.129:33843
WARNING: no policy specified for kudu/127.1.151.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab.
Entry for principal kudu/127.1.151.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab.
I20260808 06:04:52.928669 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.130:0
--local_ip_for_outbound_sockets=127.1.151.130
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:43817
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:35765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:04:53.040566 7642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:53.040825 7642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:53.040868 7642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:53.044930 7642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:53.044991 7642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:53.045058 7642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.130
I20260808 06:04:53.050489 7642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:35765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:43817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.130
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:53.051590 7642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:53.052644 7642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:53.062350 7649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:53.062386 7655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:53.062511 7642 server_base.cc:1034] running on GCE node
W20260808 06:04:53.062388 7651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:53.062891 7642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:53.063483 7642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:53.064641 7642 hybrid_clock.cc:648] HybridClock initialized: now 1786169093064632 us; error 42 us; skew 500 ppm
Aug 08 06:04:53 dist-test-slave-zj8q krb5kdc[7261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169093, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:04:53.067318 7642 init.cc:377] Logged in from keytab as kudu/127.1.151.130@KRBTEST.COM (short username kudu)
I20260808 06:04:53.068120 7642 webserver.cc:460] Webserver started at http://127.1.151.130:45193/ using document root <none> and password file <none>
I20260808 06:04:53.068637 7642 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:53.068706 7642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:53.068887 7642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:53.070874 7642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-1/data/instance:
uuid: "7e96dfeed6054e30b52928951a7ec8c8"
format_stamp: "Formatted at 2026-08-08 06:04:53 on dist-test-slave-zj8q"
server_key: "b4de3b656b0c9a8e3abac27a97429f1e"
server_key_iv: "e4fc531291a757144e85c4ffeea51317"
server_key_version: "encryptionkey@0"
I20260808 06:04:53.071403 7642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance:
uuid: "7e96dfeed6054e30b52928951a7ec8c8"
format_stamp: "Formatted at 2026-08-08 06:04:53 on dist-test-slave-zj8q"
server_key: "b4de3b656b0c9a8e3abac27a97429f1e"
server_key_iv: "e4fc531291a757144e85c4ffeea51317"
server_key_version: "encryptionkey@0"
I20260808 06:04:53.075013 7642 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 06:04:53.077320 7668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:53.078601 7642 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:04:53.078713 7642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-1/wal
uuid: "7e96dfeed6054e30b52928951a7ec8c8"
format_stamp: "Formatted at 2026-08-08 06:04:53 on dist-test-slave-zj8q"
server_key: "b4de3b656b0c9a8e3abac27a97429f1e"
server_key_iv: "e4fc531291a757144e85c4ffeea51317"
server_key_version: "encryptionkey@0"
I20260808 06:04:53.078802 7642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:53.094939 7642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:53.097882 7642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:53.098073 7642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:53.098713 7642 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:04:53.099550 7642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:04:53.099601 7642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:53.099664 7642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:04:53.099692 7642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:53.108057 7642 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.130:35233
I20260808 06:04:53.108112 7847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.130:35233 every 8 connection(s)
I20260808 06:04:53.108911 7642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
I20260808 06:04:53.115217 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 7642
I20260808 06:04:53.115343 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance
I20260808 06:04:53.115599 1630 external_mini_cluster.cc:1428] Setting key 9ef4114f4126b0a41090e850bd68b534
Aug 08 06:04:53 dist-test-slave-zj8q krb5kdc[7261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169093, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:04:53.122591 7384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:53.110850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:46701 (local address 127.1.151.190:43817)
0808 06:04:53.110982 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:53.110986 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:53.111821 (+ 835us) server_negotiation.cc:407] Connection header received
0808 06:04:53.113151 (+ 1330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:53.113155 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:53.113202 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:53.113277 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:53.114419 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:53.114943 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:53.116166 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:53.116424 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:53.116508 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:53.119232 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:53.119236 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:53.119239 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:53.119266 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:53.120698 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:53.121168 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:53.121172 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:53.121173 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:53.121259 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:53.121543 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:53.121546 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:53.121547 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:53.121759 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:53.121860 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:53.122305 (+ 445us) server_negotiation.cc:299] Negotiation successful
0808 06:04:53.122429 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 06:04:53.123343 7853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:53.111203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:43817 (local address 127.1.151.130:46701)
0808 06:04:53.111671 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:53.111715 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:53.112957 (+ 1242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:53.113426 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:53.113432 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:53.113765 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:53.114286 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:53.114302 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:53.115042 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:53.115044 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:53.116041 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:53.116052 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:53.116579 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:53.116583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:53.116799 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:53.117382 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:04:53.117398 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:53.119057 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:53.120811 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:53.120816 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:53.120827 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:53.121073 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:53.121337 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:53.121339 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:53.121340 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:53.121463 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:53.121840 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:53.121845 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:53.122056 (+ 211us) client_negotiation.cc:769] Sending connection context
0808 06:04:53.122249 (+ 193us) client_negotiation.cc:240] Negotiation successful
0808 06:04:53.122508 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":103,"threads_started":1}
I20260808 06:04:53.124433 7848 heartbeater.cc:347] Connected to a master server at 127.1.151.190:43817
I20260808 06:04:53.124666 7848 heartbeater.cc:464] Registering TS with master...
I20260808 06:04:53.125150 7848 heartbeater.cc:511] Master 127.1.151.190:43817 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.151.131@KRBTEST.COM; defaulting to no policy
I20260808 06:04:53.126281 7319 ts_manager.cc:194] Registered new tserver with Master: 7e96dfeed6054e30b52928951a7ec8c8 (127.1.151.130:35233)
I20260808 06:04:53.126871 7319 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.130@KRBTEST.COM'} at 127.1.151.130:46701
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab.
Entry for principal kudu/127.1.151.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab.
I20260808 06:04:53.143162 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.131:0
--local_ip_for_outbound_sockets=127.1.151.131
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:43817
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:35765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:04:53.255666 7861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:53.255970 7861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:53.256026 7861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:53.260174 7861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:53.260227 7861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:53.260285 7861 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.131
I20260808 06:04:53.265682 7861 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:35765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:43817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.131
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:53.266705 7861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:53.267724 7861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:53.276988 7870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:53.276993 7871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:53.277006 7875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:53.277053 7861 server_base.cc:1034] running on GCE node
I20260808 06:04:53.277685 7861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:53.278330 7861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:53.279506 7861 hybrid_clock.cc:648] HybridClock initialized: now 1786169093279482 us; error 34 us; skew 500 ppm
Aug 08 06:04:53 dist-test-slave-zj8q krb5kdc[7261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169093, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:04:53.282186 7861 init.cc:377] Logged in from keytab as kudu/127.1.151.131@KRBTEST.COM (short username kudu)
I20260808 06:04:53.283061 7861 webserver.cc:460] Webserver started at http://127.1.151.131:39151/ using document root <none> and password file <none>
I20260808 06:04:53.283587 7861 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:53.283655 7861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:53.283833 7861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:53.285945 7861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-2/data/instance:
uuid: "9e26ec247bc145619a95a8b76c3f81ea"
format_stamp: "Formatted at 2026-08-08 06:04:53 on dist-test-slave-zj8q"
server_key: "bff4698aa706ad71c937a923c6f7fae6"
server_key_iv: "48f6bed1f81c613bd9cfc665776fb74f"
server_key_version: "encryptionkey@0"
I20260808 06:04:53.286573 7861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance:
uuid: "9e26ec247bc145619a95a8b76c3f81ea"
format_stamp: "Formatted at 2026-08-08 06:04:53 on dist-test-slave-zj8q"
server_key: "bff4698aa706ad71c937a923c6f7fae6"
server_key_iv: "48f6bed1f81c613bd9cfc665776fb74f"
server_key_version: "encryptionkey@0"
I20260808 06:04:53.290519 7861 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 06:04:53.292941 7883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:53.294126 7861 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:04:53.294277 7861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-2/wal
uuid: "9e26ec247bc145619a95a8b76c3f81ea"
format_stamp: "Formatted at 2026-08-08 06:04:53 on dist-test-slave-zj8q"
server_key: "bff4698aa706ad71c937a923c6f7fae6"
server_key_iv: "48f6bed1f81c613bd9cfc665776fb74f"
server_key_version: "encryptionkey@0"
I20260808 06:04:53.294369 7861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:53.318349 7861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:53.321381 7861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:53.321539 7861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:53.322091 7861 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:04:53.322930 7861 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:04:53.322978 7861 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:53.323019 7861 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:04:53.323040 7861 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:53.331638 7861 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.131:38061
I20260808 06:04:53.331672 8053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.131:38061 every 8 connection(s)
I20260808 06:04:53.332515 7861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
I20260808 06:04:53.339623 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 7861
I20260808 06:04:53.339758 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance
I20260808 06:04:53.340096 1630 external_mini_cluster.cc:1428] Setting key 95de43a08d2c875be31d8309ecddd0cc
Aug 08 06:04:53 dist-test-slave-zj8q krb5kdc[7261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169093, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:04:53.345991 7384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:53.334341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:54607 (local address 127.1.151.190:43817)
0808 06:04:53.334479 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:53.334483 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:53.335405 (+ 922us) server_negotiation.cc:407] Connection header received
0808 06:04:53.336847 (+ 1442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:53.336850 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:53.336900 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:53.336972 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:53.338058 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:53.338709 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:53.339873 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:53.340219 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:53.340326 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:53.342968 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:53.342971 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:53.342973 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:53.342998 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:53.344350 (+ 1352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:53.344784 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:53.344786 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:53.344788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:53.344854 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:53.345093 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:53.345095 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:53.345096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:53.345277 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:53.345347 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:53.345774 (+ 427us) server_negotiation.cc:299] Negotiation successful
0808 06:04:53.345869 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 06:04:53.346863 8060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:53.334750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:43817 (local address 127.1.151.131:54607)
0808 06:04:53.335243 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:53.335294 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:53.336641 (+ 1347us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:53.337114 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:53.337119 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:53.337470 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:53.337944 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:53.337958 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:53.338826 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:53.338830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:53.339751 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:53.339764 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:53.340362 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:53.340365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:53.340549 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:53.341151 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:04:53.341167 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:53.342786 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:53.344447 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:53.344451 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:53.344463 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:53.344712 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:53.344914 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:53.344916 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:53.344917 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:53.345033 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:53.345348 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:53.345352 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:53.345579 (+ 227us) client_negotiation.cc:769] Sending connection context
0808 06:04:53.345727 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 06:04:53.345964 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":137,"threads_started":1}
I20260808 06:04:53.348049 8056 heartbeater.cc:347] Connected to a master server at 127.1.151.190:43817
I20260808 06:04:53.348286 8056 heartbeater.cc:464] Registering TS with master...
I20260808 06:04:53.348749 8056 heartbeater.cc:511] Master 127.1.151.190:43817 requested a full tablet report, sending...
I20260808 06:04:53.349732 7319 ts_manager.cc:194] Registered new tserver with Master: 9e26ec247bc145619a95a8b76c3f81ea (127.1.151.131:38061)
I20260808 06:04:53.350314 7319 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.131@KRBTEST.COM'} at 127.1.151.131:54607
I20260808 06:04:53.354063 1630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 06:04:53 dist-test-slave-zj8q krb5kdc[7261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 06:04:53 dist-test-slave-zj8q krb5kdc[7261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.129@KRBTEST.COM
I20260808 06:04:53.380890 8070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:53.371364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53272 (local address 127.1.151.129:35477)
0808 06:04:53.371642 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:53.371660 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:53.371689 (+ 29us) server_negotiation.cc:407] Connection header received
0808 06:04:53.371811 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:53.371818 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:53.372015 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:53.372164 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:53.372871 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:53.373664 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:53.374534 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:53.374911 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:53.375012 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:53.377491 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:53.377501 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:53.377515 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:53.377544 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:53.379066 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:53.379485 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:53.379491 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:53.379492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:53.379579 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:53.379838 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:53.379841 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:53.379843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:53.380112 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:53.380346 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:53.380538 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 06:04:53.380696 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":67,"threads_started":1}
Aug 08 06:04:53 dist-test-slave-zj8q krb5kdc[7261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:04:53.391601 7384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:53.383825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46446 (local address 127.1.151.190:43817)
0808 06:04:53.383958 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:53.383963 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:53.383977 (+ 14us) server_negotiation.cc:407] Connection header received
0808 06:04:53.384083 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:53.384086 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:53.384153 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:53.384220 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:53.384827 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:53.385340 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:53.386101 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:53.386381 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:53.386472 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:53.388669 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:53.388673 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:53.388676 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:53.388705 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:53.390050 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:53.390454 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:53.390458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:53.390460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:53.390536 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:53.390784 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:53.390787 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:53.390789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:53.390999 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:53.391119 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:53.391341 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 06:04:53.391448 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 06:04:53.391973 7319 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46446
I20260808 06:04:53.392598 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 7417
I20260808 06:04:53.425643 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 7642
I20260808 06:04:53.437070 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 7861
I20260808 06:04:53.445959 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 7282
2026-08-08T06:04:53Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2278 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:04:53 dist-test-slave-zj8q krb5kdc[8078](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:04:53 dist-test-slave-zj8q krb5kdc[8078](info): set up 2 sockets
Aug 08 06:04:53 dist-test-slave-zj8q krb5kdc[8078](info): commencing operation
krb5kdc: starting...
W20260808 06:04:54.600960 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:04:54 dist-test-slave-zj8q krb5kdc[8078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:04:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:04:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.151.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
I20260808 06:04:54.722890 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:37553
--webserver_interface=127.1.151.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:33737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:37553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:04:54.833266 8097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:54.833523 8097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:54.833580 8097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:54.837535 8097 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:04:54.837595 8097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:54.837621 8097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:54.837682 8097 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:04:54.837726 8097 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:04:54.842888 8097 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:33737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:37553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:37553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:54.843897 8097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:54.844894 8097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:54.853116 8107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:54.853163 8110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:54.853137 8106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:54.853263 8097 server_base.cc:1034] running on GCE node
I20260808 06:04:54.854043 8097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:54.855170 8097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:54.856331 8097 hybrid_clock.cc:648] HybridClock initialized: now 1786169094856314 us; error 26 us; skew 500 ppm
Aug 08 06:04:54 dist-test-slave-zj8q krb5kdc[8078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169094, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:04:54.859125 8097 init.cc:377] Logged in from keytab as kudu/127.1.151.190@KRBTEST.COM (short username kudu)
I20260808 06:04:54.859844 8097 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:54.859915 8097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:54.860105 8097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:54.862175 8097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "395cf84ddf28459ba9ce2abb6899be92"
format_stamp: "Formatted at 2026-08-08 06:04:54 on dist-test-slave-zj8q"
server_key: "0dc0a1e28a2dbfb3fcf790e3f9341f5d"
server_key_iv: "1252dfd3935d1bd9dcc8520d2d3c145b"
server_key_version: "encryptionkey@0"
I20260808 06:04:54.862751 8097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "395cf84ddf28459ba9ce2abb6899be92"
format_stamp: "Formatted at 2026-08-08 06:04:54 on dist-test-slave-zj8q"
server_key: "0dc0a1e28a2dbfb3fcf790e3f9341f5d"
server_key_iv: "1252dfd3935d1bd9dcc8520d2d3c145b"
server_key_version: "encryptionkey@0"
I20260808 06:04:54.866331 8097 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 06:04:54.868655 8117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:54.869817 8097 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:04:54.869930 8097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "395cf84ddf28459ba9ce2abb6899be92"
format_stamp: "Formatted at 2026-08-08 06:04:54 on dist-test-slave-zj8q"
server_key: "0dc0a1e28a2dbfb3fcf790e3f9341f5d"
server_key_iv: "1252dfd3935d1bd9dcc8520d2d3c145b"
server_key_version: "encryptionkey@0"
I20260808 06:04:54.870020 8097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:54.885802 8097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:54.888834 8097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:54.889011 8097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:54.895921 8097 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:37553
I20260808 06:04:54.896221 8209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:37553 every 8 connection(s)
I20260808 06:04:54.896595 8097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:04:54.899205 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 8097
I20260808 06:04:54.899353 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:04:54.899669 1630 external_mini_cluster.cc:1428] Setting key 27ea8bc8a0079599d6ddbac9d31e3577
I20260808 06:04:54.899677 8211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:54.907001 8211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 395cf84ddf28459ba9ce2abb6899be92: Bootstrap starting.
Aug 08 06:04:54 dist-test-slave-zj8q krb5kdc[8078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:04:54.909255 8211 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 395cf84ddf28459ba9ce2abb6899be92: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:54.910077 8211 log.cc:824] T 00000000000000000000000000000000 P 395cf84ddf28459ba9ce2abb6899be92: Log is configured to *not* fsync() on all Append() calls
I20260808 06:04:54.912313 8211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 395cf84ddf28459ba9ce2abb6899be92: No bootstrap required, opened a new log
I20260808 06:04:54.914896 8216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:54.901088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49322 (local address 127.1.151.190:37553)
0808 06:04:54.901516 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:54.901534 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:54.901576 (+ 42us) server_negotiation.cc:407] Connection header received
0808 06:04:54.902541 (+ 965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:54.902567 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:54.902988 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:54.903368 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:54.904175 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:54.905679 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:54.906570 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:54.907261 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:54.907417 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:54.909666 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:54.909673 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:54.909689 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:54.909719 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:54.911821 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:54.912233 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:54.912239 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:54.912245 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:54.912387 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:54.912660 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:54.912663 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:54.912665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:54.913161 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:54.913357 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:54.913565 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 06:04:54.913863 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":114,"threads_started":1}
I20260808 06:04:54.916428 8211 raft_consensus.cc:348] T 00000000000000000000000000000000 P 395cf84ddf28459ba9ce2abb6899be92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "395cf84ddf28459ba9ce2abb6899be92" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 37553 } }
I20260808 06:04:54.916635 8211 raft_consensus.cc:374] T 00000000000000000000000000000000 P 395cf84ddf28459ba9ce2abb6899be92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:54.916671 8211 raft_consensus.cc:729] T 00000000000000000000000000000000 P 395cf84ddf28459ba9ce2abb6899be92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 395cf84ddf28459ba9ce2abb6899be92, State: Initialized, Role: FOLLOWER
I20260808 06:04:54.917119 8211 consensus_queue.cc:260] T 00000000000000000000000000000000 P 395cf84ddf28459ba9ce2abb6899be92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "395cf84ddf28459ba9ce2abb6899be92" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 37553 } }
I20260808 06:04:54.917248 8211 raft_consensus.cc:388] T 00000000000000000000000000000000 P 395cf84ddf28459ba9ce2abb6899be92 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:04:54.917308 8211 raft_consensus.cc:482] T 00000000000000000000000000000000 P 395cf84ddf28459ba9ce2abb6899be92 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:04:54.917398 8211 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 395cf84ddf28459ba9ce2abb6899be92 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:54.918236 8211 raft_consensus.cc:504] T 00000000000000000000000000000000 P 395cf84ddf28459ba9ce2abb6899be92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "395cf84ddf28459ba9ce2abb6899be92" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 37553 } }
I20260808 06:04:54.918543 8211 leader_election.cc:302] T 00000000000000000000000000000000 P 395cf84ddf28459ba9ce2abb6899be92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 395cf84ddf28459ba9ce2abb6899be92; no voters:
I20260808 06:04:54.918812 8211 leader_election.cc:288] T 00000000000000000000000000000000 P 395cf84ddf28459ba9ce2abb6899be92 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:04:54.918912 8218 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 395cf84ddf28459ba9ce2abb6899be92 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:04:54.919371 8218 raft_consensus.cc:686] T 00000000000000000000000000000000 P 395cf84ddf28459ba9ce2abb6899be92 [term 1 LEADER]: Becoming Leader. State: Replica: 395cf84ddf28459ba9ce2abb6899be92, State: Running, Role: LEADER
I20260808 06:04:54.919677 8218 consensus_queue.cc:237] T 00000000000000000000000000000000 P 395cf84ddf28459ba9ce2abb6899be92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "395cf84ddf28459ba9ce2abb6899be92" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 37553 } }
I20260808 06:04:54.919993 8211 sys_catalog.cc:564] T 00000000000000000000000000000000 P 395cf84ddf28459ba9ce2abb6899be92 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:04:54.921090 8219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 395cf84ddf28459ba9ce2abb6899be92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "395cf84ddf28459ba9ce2abb6899be92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "395cf84ddf28459ba9ce2abb6899be92" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 37553 } } }
I20260808 06:04:54.921227 8219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 395cf84ddf28459ba9ce2abb6899be92 [sys.catalog]: This master's current role is: LEADER
I20260808 06:04:54.921644 8230 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:04:54.921643 8220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 395cf84ddf28459ba9ce2abb6899be92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 395cf84ddf28459ba9ce2abb6899be92. Latest consensus state: current_term: 1 leader_uuid: "395cf84ddf28459ba9ce2abb6899be92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "395cf84ddf28459ba9ce2abb6899be92" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 37553 } } }
I20260808 06:04:54.922407 8220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 395cf84ddf28459ba9ce2abb6899be92 [sys.catalog]: This master's current role is: LEADER
I20260808 06:04:54.924758 8230 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:04:54.929900 8230 catalog_manager.cc:1347] Generated new cluster ID: eef4f7b29c7b4fe799b572b3ea8b03fa
I20260808 06:04:54.930064 8230 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:04:54.942824 8230 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:04:54.944152 8230 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:04:54.950026 8230 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 395cf84ddf28459ba9ce2abb6899be92: Generated new TSK 0
I20260808 06:04:54.950843 8230 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.151.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab.
Entry for principal kudu/127.1.151.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab.
I20260808 06:04:54.979341 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.129:0
--local_ip_for_outbound_sockets=127.1.151.129
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:37553
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:33737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:04:55.090361 8242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:55.090606 8242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:55.090667 8242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:55.094614 8242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:55.094674 8242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:55.094751 8242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.129
I20260808 06:04:55.099956 8242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:33737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:37553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.129
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:55.101011 8242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:55.102020 8242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:55.111203 8252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:55.111302 8242 server_base.cc:1034] running on GCE node
W20260808 06:04:55.111214 8254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:55.111214 8250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:55.111832 8242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:55.112428 8242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:55.113586 8242 hybrid_clock.cc:648] HybridClock initialized: now 1786169095113559 us; error 38 us; skew 500 ppm
Aug 08 06:04:55 dist-test-slave-zj8q krb5kdc[8078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169095, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:04:55.116307 8242 init.cc:377] Logged in from keytab as kudu/127.1.151.129@KRBTEST.COM (short username kudu)
I20260808 06:04:55.117014 8242 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:55.117089 8242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:55.117286 8242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:55.119294 8242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-0/data/instance:
uuid: "aedd1f75ba264fb09990ed4ec7f93a17"
format_stamp: "Formatted at 2026-08-08 06:04:55 on dist-test-slave-zj8q"
server_key: "ece6f537fb2b9f1053f403f61ffca826"
server_key_iv: "6ce351b19a6ad784725c7f35b3671f72"
server_key_version: "encryptionkey@0"
I20260808 06:04:55.119829 8242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance:
uuid: "aedd1f75ba264fb09990ed4ec7f93a17"
format_stamp: "Formatted at 2026-08-08 06:04:55 on dist-test-slave-zj8q"
server_key: "ece6f537fb2b9f1053f403f61ffca826"
server_key_iv: "6ce351b19a6ad784725c7f35b3671f72"
server_key_version: "encryptionkey@0"
I20260808 06:04:55.123521 8242 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 06:04:55.125840 8264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:55.127007 8242 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:04:55.127115 8242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-0/wal
uuid: "aedd1f75ba264fb09990ed4ec7f93a17"
format_stamp: "Formatted at 2026-08-08 06:04:55 on dist-test-slave-zj8q"
server_key: "ece6f537fb2b9f1053f403f61ffca826"
server_key_iv: "6ce351b19a6ad784725c7f35b3671f72"
server_key_version: "encryptionkey@0"
I20260808 06:04:55.127206 8242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:55.144476 8242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:55.147677 8242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:55.147861 8242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:55.148440 8242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:04:55.149279 8242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:04:55.149329 8242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:55.149382 8242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:04:55.149412 8242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:55.157716 8242 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.129:43593
I20260808 06:04:55.157760 8452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.129:43593 every 8 connection(s)
I20260808 06:04:55.158325 8242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
I20260808 06:04:55.166383 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 8242
I20260808 06:04:55.166517 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance
I20260808 06:04:55.166857 1630 external_mini_cluster.cc:1428] Setting key c6ccdf1dd101b53a79de29dc35d6820c
Aug 08 06:04:55 dist-test-slave-zj8q krb5kdc[8078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169095, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:04:55.171463 8216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:55.160125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:48141 (local address 127.1.151.190:37553)
0808 06:04:55.160246 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:55.160249 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:55.161063 (+ 814us) server_negotiation.cc:407] Connection header received
0808 06:04:55.162374 (+ 1311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:55.162377 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:55.162426 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:55.162498 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:55.163581 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:55.164118 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:55.165202 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:55.165423 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:55.165492 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:55.168097 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:55.168100 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:55.168102 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:55.168127 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:55.169442 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:55.169915 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:55.169918 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:55.169920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:55.170001 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:55.170367 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:55.170371 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:55.170372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:55.170560 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:55.170659 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:55.171153 (+ 494us) server_negotiation.cc:299] Negotiation successful
0808 06:04:55.171293 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 06:04:55.172331 8456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:55.160441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:37553 (local address 127.1.151.129:48141)
0808 06:04:55.160907 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:55.160951 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:55.162140 (+ 1189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:55.162653 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:55.162659 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:55.163000 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:55.163452 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:55.163465 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:55.164241 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:55.164243 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:55.165107 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:55.165116 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:55.165552 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:55.165555 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:55.165718 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:55.166228 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:04:55.166248 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:55.167933 (+ 1685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:55.169552 (+ 1619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:55.169559 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:55.169573 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:55.169819 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:55.170082 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:55.170085 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:55.170087 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:55.170276 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:55.170662 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:55.170669 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:55.170914 (+ 245us) client_negotiation.cc:769] Sending connection context
0808 06:04:55.171074 (+ 160us) client_negotiation.cc:240] Negotiation successful
0808 06:04:55.171411 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":117,"threads_started":1}
I20260808 06:04:55.173604 8453 heartbeater.cc:347] Connected to a master server at 127.1.151.190:37553
I20260808 06:04:55.173893 8453 heartbeater.cc:464] Registering TS with master...
I20260808 06:04:55.174397 8453 heartbeater.cc:511] Master 127.1.151.190:37553 requested a full tablet report, sending...
I20260808 06:04:55.175899 8144 ts_manager.cc:194] Registered new tserver with Master: aedd1f75ba264fb09990ed4ec7f93a17 (127.1.151.129:43593)
WARNING: no policy specified for kudu/127.1.151.130@KRBTEST.COM; defaulting to no policy
I20260808 06:04:55.177096 8144 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.129@KRBTEST.COM'} at 127.1.151.129:48141
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab.
Entry for principal kudu/127.1.151.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab.
I20260808 06:04:55.194927 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.130:0
--local_ip_for_outbound_sockets=127.1.151.130
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:37553
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:33737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:04:55.309199 8459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:55.309437 8459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:55.309499 8459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:55.313441 8459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:55.313496 8459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:55.313639 8459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.130
I20260808 06:04:55.318753 8459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:33737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:37553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.130
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:55.319761 8459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:55.320781 8459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:55.329843 8471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:55.329851 8470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:55.329861 8473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:55.329981 8459 server_base.cc:1034] running on GCE node
I20260808 06:04:55.330497 8459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:55.331146 8459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:55.332317 8459 hybrid_clock.cc:648] HybridClock initialized: now 1786169095332299 us; error 34 us; skew 500 ppm
Aug 08 06:04:55 dist-test-slave-zj8q krb5kdc[8078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169095, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:04:55.335099 8459 init.cc:377] Logged in from keytab as kudu/127.1.151.130@KRBTEST.COM (short username kudu)
I20260808 06:04:55.335824 8459 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:55.335894 8459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:55.336112 8459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:55.338181 8459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-1/data/instance:
uuid: "0020bd6cc1dc472ebf6e5888d225b27d"
format_stamp: "Formatted at 2026-08-08 06:04:55 on dist-test-slave-zj8q"
server_key: "6f2387930005828d988acf58987df0fe"
server_key_iv: "a6c2e797d576aed86cd3ffce9744207e"
server_key_version: "encryptionkey@0"
I20260808 06:04:55.338744 8459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance:
uuid: "0020bd6cc1dc472ebf6e5888d225b27d"
format_stamp: "Formatted at 2026-08-08 06:04:55 on dist-test-slave-zj8q"
server_key: "6f2387930005828d988acf58987df0fe"
server_key_iv: "a6c2e797d576aed86cd3ffce9744207e"
server_key_version: "encryptionkey@0"
I20260808 06:04:55.342432 8459 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 06:04:55.344726 8484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:55.345865 8459 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:04:55.345970 8459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-1/wal
uuid: "0020bd6cc1dc472ebf6e5888d225b27d"
format_stamp: "Formatted at 2026-08-08 06:04:55 on dist-test-slave-zj8q"
server_key: "6f2387930005828d988acf58987df0fe"
server_key_iv: "a6c2e797d576aed86cd3ffce9744207e"
server_key_version: "encryptionkey@0"
I20260808 06:04:55.346056 8459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:55.378654 8459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:55.381587 8459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:55.381788 8459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:55.382426 8459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:04:55.383289 8459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:04:55.383339 8459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:55.383394 8459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:04:55.383419 8459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:55.391772 8459 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.130:46353
I20260808 06:04:55.391793 8676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.130:46353 every 8 connection(s)
I20260808 06:04:55.392349 8459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
Aug 08 06:04:55 dist-test-slave-zj8q krb5kdc[8078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169095, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:04:55.402190 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 8459
I20260808 06:04:55.402313 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance
I20260808 06:04:55.402698 1630 external_mini_cluster.cc:1428] Setting key 4509adb92a2fa8a7b2a0e572b257dad4
I20260808 06:04:55.405480 8216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:55.394026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:52543 (local address 127.1.151.190:37553)
0808 06:04:55.394162 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:55.394167 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:55.395036 (+ 869us) server_negotiation.cc:407] Connection header received
0808 06:04:55.396354 (+ 1318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:55.396358 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:55.396419 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:55.396492 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:55.397660 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:55.398240 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:55.399376 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:55.399609 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:55.399749 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:55.402004 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:55.402007 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:55.402009 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:55.402033 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:55.403482 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:55.404019 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:55.404022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:55.404024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:55.404132 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:55.404458 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:55.404461 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:55.404463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:55.404693 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:55.404784 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:55.405212 (+ 428us) server_negotiation.cc:299] Negotiation successful
0808 06:04:55.405328 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 06:04:55.406271 8681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:55.394427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:37553 (local address 127.1.151.130:52543)
0808 06:04:55.394884 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:55.394926 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:55.396128 (+ 1202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:55.396680 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:55.396687 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:55.397033 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:55.397502 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:55.397515 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:55.398359 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:55.398362 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:55.399251 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:55.399261 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:55.399758 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:55.399762 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:55.399940 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:55.400466 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:04:55.400482 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:55.401824 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:55.403588 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:55.403594 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:55.403607 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:55.403932 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:55.404229 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:55.404233 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:55.404235 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:55.404370 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:55.404787 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:55.404791 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:55.405006 (+ 215us) client_negotiation.cc:769] Sending connection context
0808 06:04:55.405149 (+ 143us) client_negotiation.cc:240] Negotiation successful
0808 06:04:55.405382 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":112,"threads_started":1}
I20260808 06:04:55.407312 8678 heartbeater.cc:347] Connected to a master server at 127.1.151.190:37553
I20260808 06:04:55.407531 8678 heartbeater.cc:464] Registering TS with master...
I20260808 06:04:55.407953 8678 heartbeater.cc:511] Master 127.1.151.190:37553 requested a full tablet report, sending...
I20260808 06:04:55.408962 8144 ts_manager.cc:194] Registered new tserver with Master: 0020bd6cc1dc472ebf6e5888d225b27d (127.1.151.130:46353)
I20260808 06:04:55.409538 8144 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.130@KRBTEST.COM'} at 127.1.151.130:52543
WARNING: no policy specified for kudu/127.1.151.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab.
Entry for principal kudu/127.1.151.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab.
I20260808 06:04:55.430931 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.131:0
--local_ip_for_outbound_sockets=127.1.151.131
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:37553
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:33737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:04:55.541527 8685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:55.541780 8685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:55.541824 8685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:55.545830 8685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:55.545892 8685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:55.545948 8685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.131
I20260808 06:04:55.551151 8685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:33737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:37553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.131
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:55.552121 8685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:55.553118 8685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:55.562294 8701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:55.562314 8697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:55.562314 8698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:55.562522 8685 server_base.cc:1034] running on GCE node
I20260808 06:04:55.562996 8685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:55.563616 8685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:55.564787 8685 hybrid_clock.cc:648] HybridClock initialized: now 1786169095564770 us; error 29 us; skew 500 ppm
Aug 08 06:04:55 dist-test-slave-zj8q krb5kdc[8078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169095, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:04:55.567569 8685 init.cc:377] Logged in from keytab as kudu/127.1.151.131@KRBTEST.COM (short username kudu)
I20260808 06:04:55.568288 8685 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:55.568351 8685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:55.568511 8685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:55.570557 8685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-2/data/instance:
uuid: "90b157ac58a14a5b9655edab60fbcc80"
format_stamp: "Formatted at 2026-08-08 06:04:55 on dist-test-slave-zj8q"
server_key: "7d86f7e09d0688dec32cba639fcef50d"
server_key_iv: "1f12a6b656b902e8e0d6bdaaea80056f"
server_key_version: "encryptionkey@0"
I20260808 06:04:55.571091 8685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance:
uuid: "90b157ac58a14a5b9655edab60fbcc80"
format_stamp: "Formatted at 2026-08-08 06:04:55 on dist-test-slave-zj8q"
server_key: "7d86f7e09d0688dec32cba639fcef50d"
server_key_iv: "1f12a6b656b902e8e0d6bdaaea80056f"
server_key_version: "encryptionkey@0"
I20260808 06:04:55.575006 8685 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 06:04:55.577458 8713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:55.578692 8685 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 06:04:55.578805 8685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-2/wal
uuid: "90b157ac58a14a5b9655edab60fbcc80"
format_stamp: "Formatted at 2026-08-08 06:04:55 on dist-test-slave-zj8q"
server_key: "7d86f7e09d0688dec32cba639fcef50d"
server_key_iv: "1f12a6b656b902e8e0d6bdaaea80056f"
server_key_version: "encryptionkey@0"
I20260808 06:04:55.578900 8685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:55.593703 8685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:55.596261 8685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:55.596426 8685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:55.597009 8685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:04:55.597812 8685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:04:55.597860 8685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:55.597911 8685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:04:55.597940 8685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:55.606065 8685 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.131:33571
I20260808 06:04:55.606124 8905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.131:33571 every 8 connection(s)
I20260808 06:04:55.606633 8685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
I20260808 06:04:55.607108 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 8685
I20260808 06:04:55.607235 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance
I20260808 06:04:55.607517 1630 external_mini_cluster.cc:1428] Setting key 57acddcab72ca2f4e9069049b5e4df27
Aug 08 06:04:55 dist-test-slave-zj8q krb5kdc[8078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169095, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:04:55.623646 8216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:55.608289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:33499 (local address 127.1.151.190:37553)
0808 06:04:55.608414 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:55.608418 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:55.609357 (+ 939us) server_negotiation.cc:407] Connection header received
0808 06:04:55.610769 (+ 1412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:55.610772 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:55.610828 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:55.610899 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:55.612059 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:55.612602 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:55.613730 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:55.613953 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:55.614039 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:55.620128 (+ 6089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:55.620133 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:55.620143 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:55.620172 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:55.621693 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:55.622198 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:55.622202 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:55.622204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:55.622283 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:55.622596 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:55.622599 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:55.622601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:55.622786 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:55.622872 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:55.623357 (+ 485us) server_negotiation.cc:299] Negotiation successful
0808 06:04:55.623478 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 06:04:55.624426 8910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:55.608712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:37553 (local address 127.1.151.131:33499)
0808 06:04:55.609219 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:55.609261 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:55.610556 (+ 1295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:55.611079 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:55.611086 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:55.611442 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:55.611917 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:55.611930 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:55.612723 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:55.612726 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:55.613603 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:55.613613 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:55.614091 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:55.614096 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:55.614304 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:55.618495 (+ 4191us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:04:55.618512 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:55.619963 (+ 1451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:55.621805 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:55.621810 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:55.621825 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:55.622069 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:55.622368 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:04:55.622371 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:04:55.622373 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:04:55.622506 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:04:55.622885 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:55.622892 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:55.623129 (+ 237us) client_negotiation.cc:769] Sending connection context
0808 06:04:55.623299 (+ 170us) client_negotiation.cc:240] Negotiation successful
0808 06:04:55.623617 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":133,"threads_started":1}
I20260808 06:04:55.625468 8907 heartbeater.cc:347] Connected to a master server at 127.1.151.190:37553
I20260808 06:04:55.625717 8907 heartbeater.cc:464] Registering TS with master...
I20260808 06:04:55.626176 8907 heartbeater.cc:511] Master 127.1.151.190:37553 requested a full tablet report, sending...
I20260808 06:04:55.627166 8144 ts_manager.cc:194] Registered new tserver with Master: 90b157ac58a14a5b9655edab60fbcc80 (127.1.151.131:33571)
I20260808 06:04:55.627697 8144 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.131@KRBTEST.COM'} at 127.1.151.131:33499
I20260808 06:04:55.633064 1630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 06:04:55.641352 8216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:55.634984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49338 (local address 127.1.151.190:37553)
0808 06:04:55.635097 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:55.635100 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:55.635315 (+ 215us) server_negotiation.cc:407] Connection header received
0808 06:04:55.635380 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:55.635383 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:55.635423 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:55.635484 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:55.636156 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:55.636642 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:55.637396 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:55.637590 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:55.637653 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:55.638872 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:55.638875 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:55.638877 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:04:55.638897 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:55.640114 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:55.640461 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:55.640463 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:55.640464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:55.640527 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:04:55.640710 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:04:55.640712 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:04:55.640713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:04:55.640899 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:55.640998 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:55.641132 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 06:04:55.641229 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 06:04:55.643868 8144 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49338:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 06:04:55.646804 8144 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 06:04:55.660945 8926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:55.655766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60922 (local address 127.1.151.129:43593)
0808 06:04:55.656111 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:55.656117 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:55.656135 (+ 18us) server_negotiation.cc:407] Connection header received
0808 06:04:55.656199 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:55.656204 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:55.656344 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:55.656471 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:04:55.656914 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:55.658026 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:55.659250 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:55.660050 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:55.660155 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:55.660313 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:55.660681 (+ 368us) server_negotiation.cc:299] Negotiation successful
0808 06:04:55.660776 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":81,"threads_started":1}
I20260808 06:04:55.660933 8921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:55.654772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:43593 (local address 127.0.0.1:60922)
0808 06:04:55.655086 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:55.655130 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:55.655249 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:55.656648 (+ 1399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:55.656651 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:55.656670 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:04:55.656780 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:55.656788 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:55.658183 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:55.658186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:55.659095 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:55.659106 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:55.660296 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:55.660302 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:55.660420 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:55.660529 (+ 109us) client_negotiation.cc:769] Sending connection context
0808 06:04:55.660629 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 06:04:55.660774 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":61,"thread_start_us":82,"threads_started":1}
I20260808 06:04:55.660982 8925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:55.655694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50026 (local address 127.1.151.131:33571)
0808 06:04:55.656419 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:55.656430 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:55.656452 (+ 22us) server_negotiation.cc:407] Connection header received
0808 06:04:55.656520 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:55.656525 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:55.656703 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:55.656843 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:04:55.657016 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:55.657743 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:55.659566 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:55.660257 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:55.660360 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:55.660499 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:55.660749 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 06:04:55.660850 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":126,"threads_started":1}
I20260808 06:04:55.660939 8922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:55.654877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:46353 (local address 127.0.0.1:58282)
0808 06:04:55.655452 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:55.655465 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:55.655558 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:55.656473 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:55.656479 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:55.656502 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:04:55.656658 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:55.656665 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:55.658125 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:55.658128 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:55.659317 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:55.659328 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:55.660344 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:55.660347 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:55.660490 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:55.660535 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 06:04:55.660629 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 06:04:55.660782 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"mutex_wait_us":67,"thread_start_us":104,"threads_started":1}
I20260808 06:04:55.661279 8923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:55.655466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:33571 (local address 127.0.0.1:50026)
0808 06:04:55.656136 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:55.656148 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:55.656221 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:55.656813 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:55.656816 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:55.656833 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:04:55.656927 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:55.656933 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:55.657866 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:55.657870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:55.659448 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:55.659455 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:55.660380 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:55.660384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:55.660556 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:55.660575 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 06:04:55.660634 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 06:04:55.661163 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"mutex_wait_us":533,"thread_start_us":93,"threads_started":1}
I20260808 06:04:55.661643 8924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:55.655486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58282 (local address 127.1.151.130:46353)
0808 06:04:55.655965 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:55.655976 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:55.656001 (+ 25us) server_negotiation.cc:407] Connection header received
0808 06:04:55.656080 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:55.656085 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:55.656302 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:55.656471 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:04:55.656777 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:55.657978 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:55.659458 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:55.660213 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:55.661221 (+ 1008us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:55.661377 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:55.661427 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 06:04:55.661517 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":144,"threads_started":1}
I20260808 06:04:55.664311 8355 tablet_service.cc:1467] Processing CreateTablet for tablet d1832a5988124887bd5d863030a131ff (DEFAULT_TABLE table=test-table [id=4cf872a48d3640a89b783f5d238ebdbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:04:55.664304 8787 tablet_service.cc:1467] Processing CreateTablet for tablet d1832a5988124887bd5d863030a131ff (DEFAULT_TABLE table=test-table [id=4cf872a48d3640a89b783f5d238ebdbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:04:55.664304 8569 tablet_service.cc:1467] Processing CreateTablet for tablet d1832a5988124887bd5d863030a131ff (DEFAULT_TABLE table=test-table [id=4cf872a48d3640a89b783f5d238ebdbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:04:55.665269 8355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1832a5988124887bd5d863030a131ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:55.665297 8569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1832a5988124887bd5d863030a131ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:55.665303 8787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1832a5988124887bd5d863030a131ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:55.672106 8928 tablet_bootstrap.cc:492] T d1832a5988124887bd5d863030a131ff P 90b157ac58a14a5b9655edab60fbcc80: Bootstrap starting.
I20260808 06:04:55.672328 8927 tablet_bootstrap.cc:492] T d1832a5988124887bd5d863030a131ff P aedd1f75ba264fb09990ed4ec7f93a17: Bootstrap starting.
I20260808 06:04:55.672559 8931 tablet_bootstrap.cc:492] T d1832a5988124887bd5d863030a131ff P 0020bd6cc1dc472ebf6e5888d225b27d: Bootstrap starting.
I20260808 06:04:55.674731 8931 tablet_bootstrap.cc:654] T d1832a5988124887bd5d863030a131ff P 0020bd6cc1dc472ebf6e5888d225b27d: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:55.674731 8927 tablet_bootstrap.cc:654] T d1832a5988124887bd5d863030a131ff P aedd1f75ba264fb09990ed4ec7f93a17: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:55.675648 8927 log.cc:824] T d1832a5988124887bd5d863030a131ff P aedd1f75ba264fb09990ed4ec7f93a17: Log is configured to *not* fsync() on all Append() calls
I20260808 06:04:55.675995 8931 log.cc:824] T d1832a5988124887bd5d863030a131ff P 0020bd6cc1dc472ebf6e5888d225b27d: Log is configured to *not* fsync() on all Append() calls
I20260808 06:04:55.677615 8927 tablet_bootstrap.cc:492] T d1832a5988124887bd5d863030a131ff P aedd1f75ba264fb09990ed4ec7f93a17: No bootstrap required, opened a new log
I20260808 06:04:55.677811 8927 ts_tablet_manager.cc:1397] T d1832a5988124887bd5d863030a131ff P aedd1f75ba264fb09990ed4ec7f93a17: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 06:04:55.677968 8928 tablet_bootstrap.cc:654] T d1832a5988124887bd5d863030a131ff P 90b157ac58a14a5b9655edab60fbcc80: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:55.678059 8931 tablet_bootstrap.cc:492] T d1832a5988124887bd5d863030a131ff P 0020bd6cc1dc472ebf6e5888d225b27d: No bootstrap required, opened a new log
I20260808 06:04:55.678267 8931 ts_tablet_manager.cc:1397] T d1832a5988124887bd5d863030a131ff P 0020bd6cc1dc472ebf6e5888d225b27d: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 06:04:55.679039 8928 log.cc:824] T d1832a5988124887bd5d863030a131ff P 90b157ac58a14a5b9655edab60fbcc80: Log is configured to *not* fsync() on all Append() calls
I20260808 06:04:55.680519 8928 tablet_bootstrap.cc:492] T d1832a5988124887bd5d863030a131ff P 90b157ac58a14a5b9655edab60fbcc80: No bootstrap required, opened a new log
I20260808 06:04:55.680713 8928 ts_tablet_manager.cc:1397] T d1832a5988124887bd5d863030a131ff P 90b157ac58a14a5b9655edab60fbcc80: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260808 06:04:55.681936 8927 raft_consensus.cc:348] T d1832a5988124887bd5d863030a131ff P aedd1f75ba264fb09990ed4ec7f93a17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aedd1f75ba264fb09990ed4ec7f93a17" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 43593 } } peers { permanent_uuid: "0020bd6cc1dc472ebf6e5888d225b27d" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 46353 } } peers { permanent_uuid: "90b157ac58a14a5b9655edab60fbcc80" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33571 } }
I20260808 06:04:55.681936 8931 raft_consensus.cc:348] T d1832a5988124887bd5d863030a131ff P 0020bd6cc1dc472ebf6e5888d225b27d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aedd1f75ba264fb09990ed4ec7f93a17" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 43593 } } peers { permanent_uuid: "0020bd6cc1dc472ebf6e5888d225b27d" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 46353 } } peers { permanent_uuid: "90b157ac58a14a5b9655edab60fbcc80" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33571 } }
I20260808 06:04:55.682246 8931 raft_consensus.cc:374] T d1832a5988124887bd5d863030a131ff P 0020bd6cc1dc472ebf6e5888d225b27d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:55.682277 8927 raft_consensus.cc:374] T d1832a5988124887bd5d863030a131ff P aedd1f75ba264fb09990ed4ec7f93a17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:55.682325 8931 raft_consensus.cc:729] T d1832a5988124887bd5d863030a131ff P 0020bd6cc1dc472ebf6e5888d225b27d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0020bd6cc1dc472ebf6e5888d225b27d, State: Initialized, Role: FOLLOWER
I20260808 06:04:55.682345 8927 raft_consensus.cc:729] T d1832a5988124887bd5d863030a131ff P aedd1f75ba264fb09990ed4ec7f93a17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aedd1f75ba264fb09990ed4ec7f93a17, State: Initialized, Role: FOLLOWER
I20260808 06:04:55.682840 8931 consensus_queue.cc:260] T d1832a5988124887bd5d863030a131ff P 0020bd6cc1dc472ebf6e5888d225b27d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aedd1f75ba264fb09990ed4ec7f93a17" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 43593 } } peers { permanent_uuid: "0020bd6cc1dc472ebf6e5888d225b27d" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 46353 } } peers { permanent_uuid: "90b157ac58a14a5b9655edab60fbcc80" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33571 } }
I20260808 06:04:55.682840 8927 consensus_queue.cc:260] T d1832a5988124887bd5d863030a131ff P aedd1f75ba264fb09990ed4ec7f93a17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aedd1f75ba264fb09990ed4ec7f93a17" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 43593 } } peers { permanent_uuid: "0020bd6cc1dc472ebf6e5888d225b27d" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 46353 } } peers { permanent_uuid: "90b157ac58a14a5b9655edab60fbcc80" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33571 } }
I20260808 06:04:55.683555 8678 heartbeater.cc:503] Master 127.1.151.190:37553 was elected leader, sending a full tablet report...
I20260808 06:04:55.683844 8928 raft_consensus.cc:348] T d1832a5988124887bd5d863030a131ff P 90b157ac58a14a5b9655edab60fbcc80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aedd1f75ba264fb09990ed4ec7f93a17" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 43593 } } peers { permanent_uuid: "0020bd6cc1dc472ebf6e5888d225b27d" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 46353 } } peers { permanent_uuid: "90b157ac58a14a5b9655edab60fbcc80" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33571 } }
I20260808 06:04:55.684087 8928 raft_consensus.cc:374] T d1832a5988124887bd5d863030a131ff P 90b157ac58a14a5b9655edab60fbcc80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:55.684125 8928 raft_consensus.cc:729] T d1832a5988124887bd5d863030a131ff P 90b157ac58a14a5b9655edab60fbcc80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90b157ac58a14a5b9655edab60fbcc80, State: Initialized, Role: FOLLOWER
I20260808 06:04:55.684159 8931 ts_tablet_manager.cc:1428] T d1832a5988124887bd5d863030a131ff P 0020bd6cc1dc472ebf6e5888d225b27d: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 06:04:55.684458 8928 consensus_queue.cc:260] T d1832a5988124887bd5d863030a131ff P 90b157ac58a14a5b9655edab60fbcc80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aedd1f75ba264fb09990ed4ec7f93a17" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 43593 } } peers { permanent_uuid: "0020bd6cc1dc472ebf6e5888d225b27d" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 46353 } } peers { permanent_uuid: "90b157ac58a14a5b9655edab60fbcc80" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33571 } }
I20260808 06:04:55.684906 8927 ts_tablet_manager.cc:1428] T d1832a5988124887bd5d863030a131ff P aedd1f75ba264fb09990ed4ec7f93a17: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 06:04:55.685133 8907 heartbeater.cc:503] Master 127.1.151.190:37553 was elected leader, sending a full tablet report...
I20260808 06:04:55.685232 8453 heartbeater.cc:503] Master 127.1.151.190:37553 was elected leader, sending a full tablet report...
I20260808 06:04:55.685384 8928 ts_tablet_manager.cc:1428] T d1832a5988124887bd5d863030a131ff P 90b157ac58a14a5b9655edab60fbcc80: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 06:04:55.729776 8940 raft_consensus.cc:482] T d1832a5988124887bd5d863030a131ff P 90b157ac58a14a5b9655edab60fbcc80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:04:55.729988 8940 raft_consensus.cc:504] T d1832a5988124887bd5d863030a131ff P 90b157ac58a14a5b9655edab60fbcc80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aedd1f75ba264fb09990ed4ec7f93a17" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 43593 } } peers { permanent_uuid: "0020bd6cc1dc472ebf6e5888d225b27d" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 46353 } } peers { permanent_uuid: "90b157ac58a14a5b9655edab60fbcc80" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33571 } }
I20260808 06:04:55.731498 8940 leader_election.cc:288] T d1832a5988124887bd5d863030a131ff P 90b157ac58a14a5b9655edab60fbcc80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aedd1f75ba264fb09990ed4ec7f93a17 (127.1.151.129:43593), 0020bd6cc1dc472ebf6e5888d225b27d (127.1.151.130:46353)
I20260808 06:04:55.735160 8910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:55.731682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:43593 (local address 127.1.151.131:34097)
0808 06:04:55.731794 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:55.731810 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:55.731898 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:55.732172 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:55.732175 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:55.732195 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:04:55.732313 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:55.732317 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:55.733061 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:55.733065 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:55.733986 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:55.733996 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:55.734746 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:55.734750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:55.734860 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:55.734882 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 06:04:55.734947 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 06:04:55.735021 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":3}
I20260808 06:04:55.735160 8926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:55.731813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:34097 (local address 127.1.151.129:43593)
0808 06:04:55.731933 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:55.731937 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:55.731953 (+ 16us) server_negotiation.cc:407] Connection header received
0808 06:04:55.732012 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:55.732015 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:55.732064 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:55.732145 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:04:55.732419 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:55.732953 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:55.734118 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:55.734640 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:55.734746 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:55.734801 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:55.734972 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 06:04:55.735037 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 06:04:55.735354 8924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:55.731964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:33033 (local address 127.1.151.130:46353)
0808 06:04:55.732083 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:55.732087 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:55.732282 (+ 195us) server_negotiation.cc:407] Connection header received
0808 06:04:55.732377 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:55.732380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:55.732423 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:55.732485 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:04:55.732744 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:55.733240 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:55.734354 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:55.734845 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:55.734951 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:55.735002 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:55.735183 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 06:04:55.735252 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 06:04:55.735369 8941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:55.731776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:46353 (local address 127.1.151.131:33033)
0808 06:04:55.732194 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:55.732205 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:55.732295 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:55.732502 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:55.732505 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:55.732520 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:04:55.732633 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:55.732647 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:55.733343 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:55.733347 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:55.734227 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:55.734238 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:55.734959 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:55.734962 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:55.735068 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:55.735090 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 06:04:55.735155 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 06:04:55.735220 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"mutex_wait_us":36,"thread_start_us":134,"threads_started":1}
I20260808 06:04:55.736091 8600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1832a5988124887bd5d863030a131ff" candidate_uuid: "90b157ac58a14a5b9655edab60fbcc80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0020bd6cc1dc472ebf6e5888d225b27d" is_pre_election: true
I20260808 06:04:55.736088 8383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1832a5988124887bd5d863030a131ff" candidate_uuid: "90b157ac58a14a5b9655edab60fbcc80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aedd1f75ba264fb09990ed4ec7f93a17" is_pre_election: true
I20260808 06:04:55.736418 8600 raft_consensus.cc:2454] T d1832a5988124887bd5d863030a131ff P 0020bd6cc1dc472ebf6e5888d225b27d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90b157ac58a14a5b9655edab60fbcc80 in term 0.
I20260808 06:04:55.736418 8383 raft_consensus.cc:2454] T d1832a5988124887bd5d863030a131ff P aedd1f75ba264fb09990ed4ec7f93a17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90b157ac58a14a5b9655edab60fbcc80 in term 0.
I20260808 06:04:55.736892 8717 leader_election.cc:302] T d1832a5988124887bd5d863030a131ff P 90b157ac58a14a5b9655edab60fbcc80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0020bd6cc1dc472ebf6e5888d225b27d, 90b157ac58a14a5b9655edab60fbcc80; no voters:
I20260808 06:04:55.737172 8940 raft_consensus.cc:2781] T d1832a5988124887bd5d863030a131ff P 90b157ac58a14a5b9655edab60fbcc80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 06:04:55.737258 8940 raft_consensus.cc:482] T d1832a5988124887bd5d863030a131ff P 90b157ac58a14a5b9655edab60fbcc80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 06:04:55.737309 8940 raft_consensus.cc:3037] T d1832a5988124887bd5d863030a131ff P 90b157ac58a14a5b9655edab60fbcc80 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:55.738401 8940 raft_consensus.cc:504] T d1832a5988124887bd5d863030a131ff P 90b157ac58a14a5b9655edab60fbcc80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aedd1f75ba264fb09990ed4ec7f93a17" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 43593 } } peers { permanent_uuid: "0020bd6cc1dc472ebf6e5888d225b27d" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 46353 } } peers { permanent_uuid: "90b157ac58a14a5b9655edab60fbcc80" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33571 } }
I20260808 06:04:55.738735 8940 leader_election.cc:288] T d1832a5988124887bd5d863030a131ff P 90b157ac58a14a5b9655edab60fbcc80 [CANDIDATE]: Term 1 election: Requested vote from peers aedd1f75ba264fb09990ed4ec7f93a17 (127.1.151.129:43593), 0020bd6cc1dc472ebf6e5888d225b27d (127.1.151.130:46353)
I20260808 06:04:55.739068 8383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1832a5988124887bd5d863030a131ff" candidate_uuid: "90b157ac58a14a5b9655edab60fbcc80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aedd1f75ba264fb09990ed4ec7f93a17"
I20260808 06:04:55.739087 8600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1832a5988124887bd5d863030a131ff" candidate_uuid: "90b157ac58a14a5b9655edab60fbcc80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0020bd6cc1dc472ebf6e5888d225b27d"
I20260808 06:04:55.739187 8383 raft_consensus.cc:3037] T d1832a5988124887bd5d863030a131ff P aedd1f75ba264fb09990ed4ec7f93a17 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:55.739234 8600 raft_consensus.cc:3037] T d1832a5988124887bd5d863030a131ff P 0020bd6cc1dc472ebf6e5888d225b27d [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:55.740200 8383 raft_consensus.cc:2454] T d1832a5988124887bd5d863030a131ff P aedd1f75ba264fb09990ed4ec7f93a17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90b157ac58a14a5b9655edab60fbcc80 in term 1.
I20260808 06:04:55.740307 8600 raft_consensus.cc:2454] T d1832a5988124887bd5d863030a131ff P 0020bd6cc1dc472ebf6e5888d225b27d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90b157ac58a14a5b9655edab60fbcc80 in term 1.
I20260808 06:04:55.740517 8714 leader_election.cc:302] T d1832a5988124887bd5d863030a131ff P 90b157ac58a14a5b9655edab60fbcc80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90b157ac58a14a5b9655edab60fbcc80, aedd1f75ba264fb09990ed4ec7f93a17; no voters:
I20260808 06:04:55.740695 8940 raft_consensus.cc:2781] T d1832a5988124887bd5d863030a131ff P 90b157ac58a14a5b9655edab60fbcc80 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:04:55.740908 8940 raft_consensus.cc:686] T d1832a5988124887bd5d863030a131ff P 90b157ac58a14a5b9655edab60fbcc80 [term 1 LEADER]: Becoming Leader. State: Replica: 90b157ac58a14a5b9655edab60fbcc80, State: Running, Role: LEADER
I20260808 06:04:55.741230 8940 consensus_queue.cc:237] T d1832a5988124887bd5d863030a131ff P 90b157ac58a14a5b9655edab60fbcc80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aedd1f75ba264fb09990ed4ec7f93a17" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 43593 } } peers { permanent_uuid: "0020bd6cc1dc472ebf6e5888d225b27d" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 46353 } } peers { permanent_uuid: "90b157ac58a14a5b9655edab60fbcc80" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33571 } }
I20260808 06:04:55.744472 8141 catalog_manager.cc:5495] T d1832a5988124887bd5d863030a131ff P 90b157ac58a14a5b9655edab60fbcc80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90b157ac58a14a5b9655edab60fbcc80 (127.1.151.131). New cstate: current_term: 1 leader_uuid: "90b157ac58a14a5b9655edab60fbcc80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aedd1f75ba264fb09990ed4ec7f93a17" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 43593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0020bd6cc1dc472ebf6e5888d225b27d" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 46353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90b157ac58a14a5b9655edab60fbcc80" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33571 } health_report { overall_health: HEALTHY } } }
I20260808 06:04:55.757537 8925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:55.754255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50032 (local address 127.1.151.131:33571)
0808 06:04:55.754359 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:55.754363 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:55.754376 (+ 13us) server_negotiation.cc:407] Connection header received
0808 06:04:55.754510 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:55.754513 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:55.754563 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:55.754667 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:04:55.754898 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:55.755384 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:55.756195 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:55.756416 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:55.756501 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:55.756793 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:04:55.757167 (+ 374us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:04:55.757269 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:55.757343 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 06:04:55.757397 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 06:04:55.762825 8600 raft_consensus.cc:1261] T d1832a5988124887bd5d863030a131ff P 0020bd6cc1dc472ebf6e5888d225b27d [term 1 FOLLOWER]: Refusing update from remote peer 90b157ac58a14a5b9655edab60fbcc80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:04:55.762825 8383 raft_consensus.cc:1261] T d1832a5988124887bd5d863030a131ff P aedd1f75ba264fb09990ed4ec7f93a17 [term 1 FOLLOWER]: Refusing update from remote peer 90b157ac58a14a5b9655edab60fbcc80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:04:55.763489 8943 consensus_queue.cc:1035] T d1832a5988124887bd5d863030a131ff P 90b157ac58a14a5b9655edab60fbcc80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aedd1f75ba264fb09990ed4ec7f93a17" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 43593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:04:55.763656 8940 consensus_queue.cc:1035] T d1832a5988124887bd5d863030a131ff P 90b157ac58a14a5b9655edab60fbcc80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0020bd6cc1dc472ebf6e5888d225b27d" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 46353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:04:55.773597 8947 mvcc.cc:205] Tried to move back new op lower bound from 7316148616238194688 to 7316148616158994432. Current Snapshot: MvccSnapshot[applied={T|T < 7316148616238194688}]
I20260808 06:04:55.775041 8953 mvcc.cc:205] Tried to move back new op lower bound from 7316148616238194688 to 7316148616158994432. Current Snapshot: MvccSnapshot[applied={T|T < 7316148616238194688}]
I20260808 06:04:55.783870 8142 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:04:55.784015 8142 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:04:55.786484 8142 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 395cf84ddf28459ba9ce2abb6899be92: Sending DeleteTablet for 3 replicas of tablet d1832a5988124887bd5d863030a131ff
I20260808 06:04:55.787315 8569 tablet_service.cc:1514] Processing DeleteTablet for tablet d1832a5988124887bd5d863030a131ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:04:55 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:58282
I20260808 06:04:55.787676 8787 tablet_service.cc:1514] Processing DeleteTablet for tablet d1832a5988124887bd5d863030a131ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:04:55 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:50026
I20260808 06:04:55.788130 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 8242
I20260808 06:04:55.789527 8961 tablet_replica.cc:330] T d1832a5988124887bd5d863030a131ff P 90b157ac58a14a5b9655edab60fbcc80: stopping tablet replica
I20260808 06:04:55.789906 8961 raft_consensus.cc:2229] T d1832a5988124887bd5d863030a131ff P 90b157ac58a14a5b9655edab60fbcc80 [term 1 LEADER]: Raft consensus shutting down.
I20260808 06:04:55.790392 8961 raft_consensus.cc:2258] T d1832a5988124887bd5d863030a131ff P 90b157ac58a14a5b9655edab60fbcc80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:04:55.795050 8961 ts_tablet_manager.cc:1905] T d1832a5988124887bd5d863030a131ff P 90b157ac58a14a5b9655edab60fbcc80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:04:55.797878 8961 ts_tablet_manager.cc:1918] T d1832a5988124887bd5d863030a131ff P 90b157ac58a14a5b9655edab60fbcc80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:04:55.797992 8961 log.cc:1196] T d1832a5988124887bd5d863030a131ff P 90b157ac58a14a5b9655edab60fbcc80: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786169070948234-1630-0/minicluster-data/ts-2/wal/wals/d1832a5988124887bd5d863030a131ff
I20260808 06:04:55.798350 8961 ts_tablet_manager.cc:1939] T d1832a5988124887bd5d863030a131ff P 90b157ac58a14a5b9655edab60fbcc80: Deleting consensus metadata
I20260808 06:04:55.799536 8122 catalog_manager.cc:4841] TS 90b157ac58a14a5b9655edab60fbcc80 (127.1.151.131:33571): tablet d1832a5988124887bd5d863030a131ff (table test-table [id=4cf872a48d3640a89b783f5d238ebdbc]) successfully deleted
W20260808 06:04:55.803465 8122 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.151.129:43593 (error 108)
I20260808 06:04:55.804560 8923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:55.804257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:43593 (local address 127.0.0.1:60924)
0808 06:04:55.804396 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:55.804469 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.129:43593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 06:04:55.804665 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 8459
W20260808 06:04:55.804718 8122 catalog_manager.cc:4584] TS aedd1f75ba264fb09990ed4ec7f93a17 (127.1.151.129:43593): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d1832a5988124887bd5d863030a131ff: Network error: Client connection negotiation failed: client connection to 127.1.151.129:43593: connect: Connection refused (error 111)
I20260808 06:04:55.805044 8960 tablet_replica.cc:330] T d1832a5988124887bd5d863030a131ff P 0020bd6cc1dc472ebf6e5888d225b27d: stopping tablet replica
I20260808 06:04:55.817709 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 8685
I20260808 06:04:55.820847 8923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:55.820545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:46353 (local address 127.0.0.1:58292)
0808 06:04:55.820695 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:55.820757 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.130:46353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260808 06:04:55.832046 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 8097
2026-08-08T06:04:55Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2391 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T06:04:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:04:55Z Disabled control of system clock
I20260808 06:04:55.872174 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:35421
--webserver_interface=127.1.151.190
--webserver_port=0
--builtin_ntp_servers=127.1.151.148:45947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:35421
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 06:04:55.981060 8971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:55.981349 8971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:55.981406 8971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:55.985395 8971 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:04:55.985453 8971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:55.985476 8971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:55.985495 8971 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:04:55.985517 8971 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:04:55.990244 8971 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:45947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:35421
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:35421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:55.991259 8971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:55.992595 8971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:56.001046 8986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:56.001029 8983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:56.001124 8989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:56.001715 8971 server_base.cc:1034] running on GCE node
I20260808 06:04:56.002395 8971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:56.003422 8971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:56.004585 8971 hybrid_clock.cc:648] HybridClock initialized: now 1786169096004579 us; error 39 us; skew 500 ppm
I20260808 06:04:56.006376 8971 webserver.cc:460] Webserver started at http://127.1.151.190:41363/ using document root <none> and password file <none>
I20260808 06:04:56.006906 8971 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:56.006963 8971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:56.007135 8971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:56.009069 8971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "2ea0b31c74a34f719fe39e8b2607abc3"
format_stamp: "Formatted at 2026-08-08 06:04:56 on dist-test-slave-zj8q"
server_key: "314daed82974b5e54f79cc1cb7a8fac6"
server_key_iv: "fe1d89809f76738914c392e4da28891b"
server_key_version: "encryptionkey@0"
I20260808 06:04:56.009580 8971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "2ea0b31c74a34f719fe39e8b2607abc3"
format_stamp: "Formatted at 2026-08-08 06:04:56 on dist-test-slave-zj8q"
server_key: "314daed82974b5e54f79cc1cb7a8fac6"
server_key_iv: "fe1d89809f76738914c392e4da28891b"
server_key_version: "encryptionkey@0"
I20260808 06:04:56.013182 8971 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 06:04:56.016275 9001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:56.017872 8971 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 06:04:56.018013 8971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "2ea0b31c74a34f719fe39e8b2607abc3"
format_stamp: "Formatted at 2026-08-08 06:04:56 on dist-test-slave-zj8q"
server_key: "314daed82974b5e54f79cc1cb7a8fac6"
server_key_iv: "fe1d89809f76738914c392e4da28891b"
server_key_version: "encryptionkey@0"
I20260808 06:04:56.018169 8971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:56.023519 8971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:56.024120 8971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:56.024276 8971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:56.030900 8971 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:35421
I20260808 06:04:56.030915 9079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:35421 every 8 connection(s)
I20260808 06:04:56.031765 8971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:04:56.034974 9080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:56.037369 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 8971
I20260808 06:04:56.037489 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:04:56.037794 1630 external_mini_cluster.cc:1428] Setting key 1b6784f2035e9fcf6553e6369d82d0ec
I20260808 06:04:56.040802 9080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ea0b31c74a34f719fe39e8b2607abc3: Bootstrap starting.
I20260808 06:04:56.043042 9080 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ea0b31c74a34f719fe39e8b2607abc3: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:56.043247 9086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:56.039247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45462 (local address 127.1.151.190:35421)
0808 06:04:56.039894 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:56.039914 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:56.039949 (+ 35us) server_negotiation.cc:407] Connection header received
0808 06:04:56.040550 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:56.040566 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:56.040920 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:56.041228 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:56.041981 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:56.041986 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:56.041997 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:04:56.042008 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:56.042060 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:04:56.042077 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:56.042221 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:56.042321 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 06:04:56.042561 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":137,"threads_started":1}
I20260808 06:04:56.043772 9080 log.cc:824] T 00000000000000000000000000000000 P 2ea0b31c74a34f719fe39e8b2607abc3: Log is configured to *not* fsync() on all Append() calls
I20260808 06:04:56.045645 9080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ea0b31c74a34f719fe39e8b2607abc3: No bootstrap required, opened a new log
I20260808 06:04:56.049204 9080 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ea0b31c74a34f719fe39e8b2607abc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea0b31c74a34f719fe39e8b2607abc3" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35421 } }
I20260808 06:04:56.049419 9080 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ea0b31c74a34f719fe39e8b2607abc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:56.049475 9080 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ea0b31c74a34f719fe39e8b2607abc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ea0b31c74a34f719fe39e8b2607abc3, State: Initialized, Role: FOLLOWER
I20260808 06:04:56.049860 9080 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ea0b31c74a34f719fe39e8b2607abc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea0b31c74a34f719fe39e8b2607abc3" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35421 } }
I20260808 06:04:56.049981 9080 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ea0b31c74a34f719fe39e8b2607abc3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:04:56.050040 9080 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ea0b31c74a34f719fe39e8b2607abc3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:04:56.050127 9080 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ea0b31c74a34f719fe39e8b2607abc3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:56.051175 9080 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ea0b31c74a34f719fe39e8b2607abc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea0b31c74a34f719fe39e8b2607abc3" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35421 } }
I20260808 06:04:56.051492 9080 leader_election.cc:302] T 00000000000000000000000000000000 P 2ea0b31c74a34f719fe39e8b2607abc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ea0b31c74a34f719fe39e8b2607abc3; no voters:
I20260808 06:04:56.051795 9080 leader_election.cc:288] T 00000000000000000000000000000000 P 2ea0b31c74a34f719fe39e8b2607abc3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:04:56.051887 9088 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ea0b31c74a34f719fe39e8b2607abc3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:04:56.052091 9088 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ea0b31c74a34f719fe39e8b2607abc3 [term 1 LEADER]: Becoming Leader. State: Replica: 2ea0b31c74a34f719fe39e8b2607abc3, State: Running, Role: LEADER
I20260808 06:04:56.052403 9088 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ea0b31c74a34f719fe39e8b2607abc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea0b31c74a34f719fe39e8b2607abc3" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35421 } }
I20260808 06:04:56.052994 9080 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ea0b31c74a34f719fe39e8b2607abc3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:04:56.053985 9089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ea0b31c74a34f719fe39e8b2607abc3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ea0b31c74a34f719fe39e8b2607abc3. Latest consensus state: current_term: 1 leader_uuid: "2ea0b31c74a34f719fe39e8b2607abc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea0b31c74a34f719fe39e8b2607abc3" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35421 } } }
I20260808 06:04:56.054129 9089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ea0b31c74a34f719fe39e8b2607abc3 [sys.catalog]: This master's current role is: LEADER
I20260808 06:04:56.054440 9092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ea0b31c74a34f719fe39e8b2607abc3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ea0b31c74a34f719fe39e8b2607abc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea0b31c74a34f719fe39e8b2607abc3" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35421 } } }
I20260808 06:04:56.054535 9092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ea0b31c74a34f719fe39e8b2607abc3 [sys.catalog]: This master's current role is: LEADER
I20260808 06:04:56.054555 9103 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:04:56.057621 9103 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:04:56.062732 9103 catalog_manager.cc:1347] Generated new cluster ID: 0bd6800b1ef94b38a01d4e8356ce16c8
I20260808 06:04:56.062795 9103 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:04:56.079039 9103 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:04:56.079368 9103 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:04:56.087940 9103 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ea0b31c74a34f719fe39e8b2607abc3: Generated new TSK 0
I20260808 06:04:56.088661 9103 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 06:04:56.091177 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.129:0
--local_ip_for_outbound_sockets=127.1.151.129
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:35421
--builtin_ntp_servers=127.1.151.148:45947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 06:04:56.203996 9118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:56.204237 9118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:56.204281 9118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:56.208313 9118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:56.208364 9118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:56.208426 9118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.129
I20260808 06:04:56.213284 9118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:45947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:35421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.129
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:56.214452 9118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:56.215871 9118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:56.225558 9127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:56.225544 9130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:56.225595 9118 server_base.cc:1034] running on GCE node
W20260808 06:04:56.225507 9124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:56.226125 9118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:56.226788 9118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:56.227955 9118 hybrid_clock.cc:648] HybridClock initialized: now 1786169096227937 us; error 29 us; skew 500 ppm
I20260808 06:04:56.229758 9118 webserver.cc:460] Webserver started at http://127.1.151.129:45843/ using document root <none> and password file <none>
I20260808 06:04:56.230360 9118 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:56.230429 9118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:56.230649 9118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:56.232632 9118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-0/data/instance:
uuid: "d509319a6f964980acdf6d7a77baf7ca"
format_stamp: "Formatted at 2026-08-08 06:04:56 on dist-test-slave-zj8q"
server_key: "0dd3b7f23951ba0ba2bb39d32fea23cd"
server_key_iv: "9c8eec791c9a9359cb0dd8756744f0c5"
server_key_version: "encryptionkey@0"
I20260808 06:04:56.233151 9118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance:
uuid: "d509319a6f964980acdf6d7a77baf7ca"
format_stamp: "Formatted at 2026-08-08 06:04:56 on dist-test-slave-zj8q"
server_key: "0dd3b7f23951ba0ba2bb39d32fea23cd"
server_key_iv: "9c8eec791c9a9359cb0dd8756744f0c5"
server_key_version: "encryptionkey@0"
I20260808 06:04:56.237306 9118 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260808 06:04:56.239900 9146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:56.241098 9118 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 06:04:56.241201 9118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-0/wal
uuid: "d509319a6f964980acdf6d7a77baf7ca"
format_stamp: "Formatted at 2026-08-08 06:04:56 on dist-test-slave-zj8q"
server_key: "0dd3b7f23951ba0ba2bb39d32fea23cd"
server_key_iv: "9c8eec791c9a9359cb0dd8756744f0c5"
server_key_version: "encryptionkey@0"
I20260808 06:04:56.241297 9118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:56.245885 9118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:56.246482 9118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:56.246644 9118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:56.247175 9118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:04:56.248072 9118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:04:56.248121 9118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:56.248163 9118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:04:56.248191 9118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:56.256579 9118 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.129:46611
I20260808 06:04:56.256603 9322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.129:46611 every 8 connection(s)
I20260808 06:04:56.257457 9118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
I20260808 06:04:56.258230 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 9118
I20260808 06:04:56.258358 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance
I20260808 06:04:56.258673 1630 external_mini_cluster.cc:1428] Setting key 27f99dd8137b9021889113f905c009e7
I20260808 06:04:56.260691 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.130:0
--local_ip_for_outbound_sockets=127.1.151.130
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:35421
--builtin_ntp_servers=127.1.151.148:45947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 06:04:56.263927 9086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:56.260866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:42741 (local address 127.1.151.190:35421)
0808 06:04:56.260980 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:56.260985 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:56.260999 (+ 14us) server_negotiation.cc:407] Connection header received
0808 06:04:56.262194 (+ 1195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:56.262197 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:56.262241 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:56.262317 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:56.263459 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:56.263463 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:56.263466 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:04:56.263479 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:56.263498 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:04:56.263506 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:56.263584 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:56.263757 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 06:04:56.263800 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 06:04:56.264743 9325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:56.259838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35421 (local address 127.1.151.129:42741)
0808 06:04:56.260590 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:56.260646 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:56.261951 (+ 1305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:56.262550 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:56.262557 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:56.262656 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:56.263181 (+ 525us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 06:04:56.263182 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:56.263222 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 06:04:56.263295 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:56.263597 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:56.263606 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:56.263611 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 06:04:56.263776 (+ 165us) client_negotiation.cc:240] Negotiation successful
0808 06:04:56.263975 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":150,"threads_started":1}
I20260808 06:04:56.265921 9323 heartbeater.cc:347] Connected to a master server at 127.1.151.190:35421
I20260808 06:04:56.266204 9323 heartbeater.cc:464] Registering TS with master...
I20260808 06:04:56.266690 9323 heartbeater.cc:511] Master 127.1.151.190:35421 requested a full tablet report, sending...
I20260808 06:04:56.268286 9023 ts_manager.cc:194] Registered new tserver with Master: d509319a6f964980acdf6d7a77baf7ca (127.1.151.129:46611)
W20260808 06:04:56.372494 9326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:56.372761 9326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:56.372814 9326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:56.376811 9326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:56.376871 9326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:56.376945 9326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.130
I20260808 06:04:56.381629 9326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:45947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:35421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.130
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:56.382720 9326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:56.384095 9326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 06:04:56.393334 9326 server_base.cc:1034] running on GCE node
W20260808 06:04:56.393301 9336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:56.393299 9335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:56.393306 9339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:56.393972 9326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:56.394629 9326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:56.395784 9326 hybrid_clock.cc:648] HybridClock initialized: now 1786169096395759 us; error 39 us; skew 500 ppm
I20260808 06:04:56.397545 9326 webserver.cc:460] Webserver started at http://127.1.151.130:45371/ using document root <none> and password file <none>
I20260808 06:04:56.398090 9326 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:56.398177 9326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:56.398397 9326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:56.400382 9326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-1/data/instance:
uuid: "286752113f2145c9a37a31e2c1dbc2e3"
format_stamp: "Formatted at 2026-08-08 06:04:56 on dist-test-slave-zj8q"
server_key: "3683cf3682bc3a7cee4a9ca167d207b5"
server_key_iv: "57631d163cb3c5ebab517b902b7915b4"
server_key_version: "encryptionkey@0"
I20260808 06:04:56.400911 9326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance:
uuid: "286752113f2145c9a37a31e2c1dbc2e3"
format_stamp: "Formatted at 2026-08-08 06:04:56 on dist-test-slave-zj8q"
server_key: "3683cf3682bc3a7cee4a9ca167d207b5"
server_key_iv: "57631d163cb3c5ebab517b902b7915b4"
server_key_version: "encryptionkey@0"
I20260808 06:04:56.404387 9326 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260808 06:04:56.406694 9351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:56.407835 9326 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:04:56.407943 9326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-1/wal
uuid: "286752113f2145c9a37a31e2c1dbc2e3"
format_stamp: "Formatted at 2026-08-08 06:04:56 on dist-test-slave-zj8q"
server_key: "3683cf3682bc3a7cee4a9ca167d207b5"
server_key_iv: "57631d163cb3c5ebab517b902b7915b4"
server_key_version: "encryptionkey@0"
I20260808 06:04:56.408033 9326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:56.412376 9326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:56.412948 9326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:56.413100 9326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:56.413630 9326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:04:56.414566 9326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:04:56.414615 9326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:56.414676 9326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:04:56.414701 9326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:56.423132 9326 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.130:41011
I20260808 06:04:56.423152 9538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.130:41011 every 8 connection(s)
I20260808 06:04:56.423991 9326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
I20260808 06:04:56.427631 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 9326
I20260808 06:04:56.427767 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance
I20260808 06:04:56.428038 1630 external_mini_cluster.cc:1428] Setting key 1ca9e51ca8961056c460b68b4df82d9f
I20260808 06:04:56.429839 9086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:56.425709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:55475 (local address 127.1.151.190:35421)
0808 06:04:56.425846 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:56.425851 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:56.426866 (+ 1015us) server_negotiation.cc:407] Connection header received
0808 06:04:56.428223 (+ 1357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:56.428227 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:56.428273 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:56.428350 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:56.429402 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:56.429406 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:56.429408 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:04:56.429421 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:56.429444 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:04:56.429452 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:56.429526 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:56.429659 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 06:04:56.429703 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 06:04:56.429919 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.131:0
--local_ip_for_outbound_sockets=127.1.151.131
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:35421
--builtin_ntp_servers=127.1.151.148:45947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 06:04:56.430759 9541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:56.426046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35421 (local address 127.1.151.130:55475)
0808 06:04:56.426728 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:56.426768 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:56.427994 (+ 1226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:56.428506 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:56.428513 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:56.428597 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:56.429126 (+ 529us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 06:04:56.429128 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:56.429192 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 06:04:56.429266 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:56.429521 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:56.429529 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:56.429535 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 06:04:56.429687 (+ 152us) client_negotiation.cc:240] Negotiation successful
0808 06:04:56.429926 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":125,"threads_started":1}
I20260808 06:04:56.432411 9539 heartbeater.cc:347] Connected to a master server at 127.1.151.190:35421
I20260808 06:04:56.432658 9539 heartbeater.cc:464] Registering TS with master...
I20260808 06:04:56.433094 9539 heartbeater.cc:511] Master 127.1.151.190:35421 requested a full tablet report, sending...
I20260808 06:04:56.434026 9023 ts_manager.cc:194] Registered new tserver with Master: 286752113f2145c9a37a31e2c1dbc2e3 (127.1.151.130:41011)
W20260808 06:04:56.540999 9542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:56.541247 9542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:56.541285 9542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:56.545279 9542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:56.545337 9542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:56.545410 9542 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.131
I20260808 06:04:56.550145 9542 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:45947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:35421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.131
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:56.551198 9542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:56.552548 9542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:56.562204 9550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:56.562250 9548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:56.562234 9555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:56.562573 9542 server_base.cc:1034] running on GCE node
I20260808 06:04:56.562960 9542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:56.563612 9542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:56.564797 9542 hybrid_clock.cc:648] HybridClock initialized: now 1786169096564780 us; error 32 us; skew 500 ppm
I20260808 06:04:56.566654 9542 webserver.cc:460] Webserver started at http://127.1.151.131:36593/ using document root <none> and password file <none>
I20260808 06:04:56.567256 9542 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:56.567323 9542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:56.567512 9542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:56.569525 9542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-2/data/instance:
uuid: "41a5e493e78f4c9c8f46b61e4310d618"
format_stamp: "Formatted at 2026-08-08 06:04:56 on dist-test-slave-zj8q"
server_key: "b0dd03ddd4a6ffc98108f7087b52be73"
server_key_iv: "1ae40a689a5ecf5d4d7434c7c88ff230"
server_key_version: "encryptionkey@0"
I20260808 06:04:56.570076 9542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance:
uuid: "41a5e493e78f4c9c8f46b61e4310d618"
format_stamp: "Formatted at 2026-08-08 06:04:56 on dist-test-slave-zj8q"
server_key: "b0dd03ddd4a6ffc98108f7087b52be73"
server_key_iv: "1ae40a689a5ecf5d4d7434c7c88ff230"
server_key_version: "encryptionkey@0"
I20260808 06:04:56.573794 9542 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 06:04:56.576367 9569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:56.577714 9542 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:04:56.577840 9542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-2/wal
uuid: "41a5e493e78f4c9c8f46b61e4310d618"
format_stamp: "Formatted at 2026-08-08 06:04:56 on dist-test-slave-zj8q"
server_key: "b0dd03ddd4a6ffc98108f7087b52be73"
server_key_iv: "1ae40a689a5ecf5d4d7434c7c88ff230"
server_key_version: "encryptionkey@0"
I20260808 06:04:56.577940 9542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:56.582511 9542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:56.583101 9542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:56.583267 9542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:56.583806 9542 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:04:56.584707 9542 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:04:56.584753 9542 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:56.584805 9542 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:04:56.584833 9542 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:56.593456 9542 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.131:38891
I20260808 06:04:56.593498 9761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.131:38891 every 8 connection(s)
I20260808 06:04:56.594377 9542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
I20260808 06:04:56.597658 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 9542
I20260808 06:04:56.597783 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance
I20260808 06:04:56.598061 1630 external_mini_cluster.cc:1428] Setting key 9af729f7fe8cd5e3ab22dd2251789459
I20260808 06:04:56.600253 9086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:56.596065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:33971 (local address 127.1.151.190:35421)
0808 06:04:56.596199 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:56.596203 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:56.597218 (+ 1015us) server_negotiation.cc:407] Connection header received
0808 06:04:56.598605 (+ 1387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:56.598609 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:56.598654 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:56.598729 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:56.599790 (+ 1061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:56.599794 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:56.599796 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:04:56.599810 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:56.599833 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:04:56.599842 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:56.599928 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:56.600071 (+ 143us) server_negotiation.cc:299] Negotiation successful
0808 06:04:56.600116 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 06:04:56.601049 9766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:56.596438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35421 (local address 127.1.151.131:33971)
0808 06:04:56.597070 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:56.597109 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:56.598399 (+ 1290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:56.598894 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:56.598901 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:56.598988 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:56.599536 (+ 548us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 06:04:56.599538 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:56.599576 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 06:04:56.599656 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:56.599937 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:56.599947 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:56.599954 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 06:04:56.600107 (+ 153us) client_negotiation.cc:240] Negotiation successful
0808 06:04:56.600345 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":109,"threads_started":1}
I20260808 06:04:56.602007 9762 heartbeater.cc:347] Connected to a master server at 127.1.151.190:35421
I20260808 06:04:56.602252 9762 heartbeater.cc:464] Registering TS with master...
I20260808 06:04:56.602654 9762 heartbeater.cc:511] Master 127.1.151.190:35421 requested a full tablet report, sending...
I20260808 06:04:56.603600 9023 ts_manager.cc:194] Registered new tserver with Master: 41a5e493e78f4c9c8f46b61e4310d618 (127.1.151.131:38891)
I20260808 06:04:56.611732 1630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 06:04:56.615098 9086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:56.613608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45474 (local address 127.1.151.190:35421)
0808 06:04:56.613723 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:56.613726 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:56.613898 (+ 172us) server_negotiation.cc:407] Connection header received
0808 06:04:56.613994 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:56.613996 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:56.614028 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:56.614094 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:56.614820 (+ 726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:56.614823 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:56.614825 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:04:56.614834 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:56.614852 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:04:56.614859 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:56.614921 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:56.614967 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 06:04:56.614999 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 06:04:56.617702 9023 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45474:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 06:04:56.620676 9023 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 06:04:56.631847 9780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:56.628738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:41011 (local address 127.0.0.1:49442)
0808 06:04:56.629042 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:56.629093 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:56.629196 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:56.630308 (+ 1112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:56.630314 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:56.630393 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:56.630908 (+ 515us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 06:04:56.630910 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:56.630931 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 06:04:56.631000 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:56.631512 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:56.631522 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:56.631527 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 06:04:56.631641 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 06:04:56.631711 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":74,"threads_started":1}
I20260808 06:04:56.631914 9781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:56.628917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49442 (local address 127.1.151.130:41011)
0808 06:04:56.629396 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:56.629409 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:56.629435 (+ 26us) server_negotiation.cc:407] Connection header received
0808 06:04:56.629494 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:56.629500 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:56.629773 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:56.629922 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:56.631174 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:56.631181 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:56.631193 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:04:56.631216 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:56.631264 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:04:56.631309 (+ 45us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:56.631431 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:56.631669 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 06:04:56.631749 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":154,"threads_started":1}
I20260808 06:04:56.632179 9788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:56.629829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:38891 (local address 127.0.0.1:33122)
0808 06:04:56.630137 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:56.630169 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:56.630254 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:56.631001 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:56.631004 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:56.631020 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:56.631587 (+ 567us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 06:04:56.631590 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:56.631609 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 06:04:56.631638 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:56.631977 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:56.631980 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:56.631984 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 06:04:56.632037 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 06:04:56.632070 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":101,"threads_started":1}
I20260808 06:04:56.632879 9789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:56.630279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33122 (local address 127.1.151.131:38891)
0808 06:04:56.630619 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:56.630631 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:56.630654 (+ 23us) server_negotiation.cc:407] Connection header received
0808 06:04:56.630704 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:56.630709 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:56.630873 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:56.631000 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:56.631769 (+ 769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:56.631775 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:56.631786 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:04:56.631798 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:56.631838 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:04:56.631873 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:56.632592 (+ 719us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:56.632651 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 06:04:56.632720 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":144,"threads_started":1}
I20260808 06:04:56.633816 9782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:56.629494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:46611 (local address 127.0.0.1:50632)
0808 06:04:56.629719 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:56.629732 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:56.629810 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:56.631186 (+ 1376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:56.631189 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:56.631204 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:56.631552 (+ 348us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 06:04:56.631553 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:56.631564 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 06:04:56.631600 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:56.633576 (+ 1976us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:56.633579 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:56.633583 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 06:04:56.633673 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 06:04:56.633706 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":89,"threads_started":1}
I20260808 06:04:56.634315 9783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:56.629692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50632 (local address 127.1.151.129:46611)
0808 06:04:56.630766 (+ 1074us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:56.630776 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:56.630802 (+ 26us) server_negotiation.cc:407] Connection header received
0808 06:04:56.630857 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:56.630863 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:56.631045 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:56.631418 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:56.633390 (+ 1972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:56.633396 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:56.633405 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:04:56.633414 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:56.633450 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:04:56.633472 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:56.633579 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:56.634063 (+ 484us) server_negotiation.cc:299] Negotiation successful
0808 06:04:56.634132 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":976,"thread_start_us":148,"threads_started":1}
I20260808 06:04:56.635226 9421 tablet_service.cc:1467] Processing CreateTablet for tablet 42a3dfb726e940f3adea14faa15f3ef4 (DEFAULT_TABLE table=test-table [id=1ee5289c1dbc4eb9a3cacb96cbcfd7e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:04:56.635349 9658 tablet_service.cc:1467] Processing CreateTablet for tablet 42a3dfb726e940f3adea14faa15f3ef4 (DEFAULT_TABLE table=test-table [id=1ee5289c1dbc4eb9a3cacb96cbcfd7e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:04:56.636305 9421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42a3dfb726e940f3adea14faa15f3ef4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:56.636379 9658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42a3dfb726e940f3adea14faa15f3ef4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:56.636876 9225 tablet_service.cc:1467] Processing CreateTablet for tablet 42a3dfb726e940f3adea14faa15f3ef4 (DEFAULT_TABLE table=test-table [id=1ee5289c1dbc4eb9a3cacb96cbcfd7e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:04:56.637797 9225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42a3dfb726e940f3adea14faa15f3ef4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:56.642004 9790 tablet_bootstrap.cc:492] T 42a3dfb726e940f3adea14faa15f3ef4 P 286752113f2145c9a37a31e2c1dbc2e3: Bootstrap starting.
I20260808 06:04:56.643713 9791 tablet_bootstrap.cc:492] T 42a3dfb726e940f3adea14faa15f3ef4 P 41a5e493e78f4c9c8f46b61e4310d618: Bootstrap starting.
I20260808 06:04:56.644620 9792 tablet_bootstrap.cc:492] T 42a3dfb726e940f3adea14faa15f3ef4 P d509319a6f964980acdf6d7a77baf7ca: Bootstrap starting.
I20260808 06:04:56.644959 9790 tablet_bootstrap.cc:654] T 42a3dfb726e940f3adea14faa15f3ef4 P 286752113f2145c9a37a31e2c1dbc2e3: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:56.645881 9790 log.cc:824] T 42a3dfb726e940f3adea14faa15f3ef4 P 286752113f2145c9a37a31e2c1dbc2e3: Log is configured to *not* fsync() on all Append() calls
I20260808 06:04:56.645967 9791 tablet_bootstrap.cc:654] T 42a3dfb726e940f3adea14faa15f3ef4 P 41a5e493e78f4c9c8f46b61e4310d618: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:56.646672 9792 tablet_bootstrap.cc:654] T 42a3dfb726e940f3adea14faa15f3ef4 P d509319a6f964980acdf6d7a77baf7ca: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:56.646744 9791 log.cc:824] T 42a3dfb726e940f3adea14faa15f3ef4 P 41a5e493e78f4c9c8f46b61e4310d618: Log is configured to *not* fsync() on all Append() calls
I20260808 06:04:56.647447 9792 log.cc:824] T 42a3dfb726e940f3adea14faa15f3ef4 P d509319a6f964980acdf6d7a77baf7ca: Log is configured to *not* fsync() on all Append() calls
I20260808 06:04:56.647929 9790 tablet_bootstrap.cc:492] T 42a3dfb726e940f3adea14faa15f3ef4 P 286752113f2145c9a37a31e2c1dbc2e3: No bootstrap required, opened a new log
I20260808 06:04:56.648128 9790 ts_tablet_manager.cc:1397] T 42a3dfb726e940f3adea14faa15f3ef4 P 286752113f2145c9a37a31e2c1dbc2e3: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 06:04:56.648799 9791 tablet_bootstrap.cc:492] T 42a3dfb726e940f3adea14faa15f3ef4 P 41a5e493e78f4c9c8f46b61e4310d618: No bootstrap required, opened a new log
I20260808 06:04:56.648985 9791 ts_tablet_manager.cc:1397] T 42a3dfb726e940f3adea14faa15f3ef4 P 41a5e493e78f4c9c8f46b61e4310d618: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 06:04:56.649137 9792 tablet_bootstrap.cc:492] T 42a3dfb726e940f3adea14faa15f3ef4 P d509319a6f964980acdf6d7a77baf7ca: No bootstrap required, opened a new log
I20260808 06:04:56.649312 9792 ts_tablet_manager.cc:1397] T 42a3dfb726e940f3adea14faa15f3ef4 P d509319a6f964980acdf6d7a77baf7ca: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 06:04:56.652693 9792 raft_consensus.cc:348] T 42a3dfb726e940f3adea14faa15f3ef4 P d509319a6f964980acdf6d7a77baf7ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "286752113f2145c9a37a31e2c1dbc2e3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 41011 } } peers { permanent_uuid: "d509319a6f964980acdf6d7a77baf7ca" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 46611 } } peers { permanent_uuid: "41a5e493e78f4c9c8f46b61e4310d618" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 38891 } }
I20260808 06:04:56.652909 9792 raft_consensus.cc:374] T 42a3dfb726e940f3adea14faa15f3ef4 P d509319a6f964980acdf6d7a77baf7ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:56.652979 9792 raft_consensus.cc:729] T 42a3dfb726e940f3adea14faa15f3ef4 P d509319a6f964980acdf6d7a77baf7ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d509319a6f964980acdf6d7a77baf7ca, State: Initialized, Role: FOLLOWER
I20260808 06:04:56.652846 9790 raft_consensus.cc:348] T 42a3dfb726e940f3adea14faa15f3ef4 P 286752113f2145c9a37a31e2c1dbc2e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "286752113f2145c9a37a31e2c1dbc2e3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 41011 } } peers { permanent_uuid: "d509319a6f964980acdf6d7a77baf7ca" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 46611 } } peers { permanent_uuid: "41a5e493e78f4c9c8f46b61e4310d618" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 38891 } }
I20260808 06:04:56.653126 9790 raft_consensus.cc:374] T 42a3dfb726e940f3adea14faa15f3ef4 P 286752113f2145c9a37a31e2c1dbc2e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:56.653198 9790 raft_consensus.cc:729] T 42a3dfb726e940f3adea14faa15f3ef4 P 286752113f2145c9a37a31e2c1dbc2e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 286752113f2145c9a37a31e2c1dbc2e3, State: Initialized, Role: FOLLOWER
I20260808 06:04:56.653223 9791 raft_consensus.cc:348] T 42a3dfb726e940f3adea14faa15f3ef4 P 41a5e493e78f4c9c8f46b61e4310d618 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "286752113f2145c9a37a31e2c1dbc2e3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 41011 } } peers { permanent_uuid: "d509319a6f964980acdf6d7a77baf7ca" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 46611 } } peers { permanent_uuid: "41a5e493e78f4c9c8f46b61e4310d618" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 38891 } }
I20260808 06:04:56.653450 9792 consensus_queue.cc:260] T 42a3dfb726e940f3adea14faa15f3ef4 P d509319a6f964980acdf6d7a77baf7ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "286752113f2145c9a37a31e2c1dbc2e3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 41011 } } peers { permanent_uuid: "d509319a6f964980acdf6d7a77baf7ca" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 46611 } } peers { permanent_uuid: "41a5e493e78f4c9c8f46b61e4310d618" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 38891 } }
I20260808 06:04:56.653537 9791 raft_consensus.cc:374] T 42a3dfb726e940f3adea14faa15f3ef4 P 41a5e493e78f4c9c8f46b61e4310d618 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:56.653597 9791 raft_consensus.cc:729] T 42a3dfb726e940f3adea14faa15f3ef4 P 41a5e493e78f4c9c8f46b61e4310d618 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41a5e493e78f4c9c8f46b61e4310d618, State: Initialized, Role: FOLLOWER
I20260808 06:04:56.653671 9790 consensus_queue.cc:260] T 42a3dfb726e940f3adea14faa15f3ef4 P 286752113f2145c9a37a31e2c1dbc2e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "286752113f2145c9a37a31e2c1dbc2e3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 41011 } } peers { permanent_uuid: "d509319a6f964980acdf6d7a77baf7ca" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 46611 } } peers { permanent_uuid: "41a5e493e78f4c9c8f46b61e4310d618" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 38891 } }
I20260808 06:04:56.654224 9323 heartbeater.cc:503] Master 127.1.151.190:35421 was elected leader, sending a full tablet report...
I20260808 06:04:56.654508 9792 ts_tablet_manager.cc:1428] T 42a3dfb726e940f3adea14faa15f3ef4 P d509319a6f964980acdf6d7a77baf7ca: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 06:04:56.654764 9790 ts_tablet_manager.cc:1428] T 42a3dfb726e940f3adea14faa15f3ef4 P 286752113f2145c9a37a31e2c1dbc2e3: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 06:04:56.654706 9791 consensus_queue.cc:260] T 42a3dfb726e940f3adea14faa15f3ef4 P 41a5e493e78f4c9c8f46b61e4310d618 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "286752113f2145c9a37a31e2c1dbc2e3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 41011 } } peers { permanent_uuid: "d509319a6f964980acdf6d7a77baf7ca" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 46611 } } peers { permanent_uuid: "41a5e493e78f4c9c8f46b61e4310d618" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 38891 } }
I20260808 06:04:56.655138 9539 heartbeater.cc:503] Master 127.1.151.190:35421 was elected leader, sending a full tablet report...
I20260808 06:04:56.655527 9762 heartbeater.cc:503] Master 127.1.151.190:35421 was elected leader, sending a full tablet report...
I20260808 06:04:56.655629 9791 ts_tablet_manager.cc:1428] T 42a3dfb726e940f3adea14faa15f3ef4 P 41a5e493e78f4c9c8f46b61e4310d618: Time spent starting tablet: real 0.007s user 0.002s sys 0.005s
W20260808 06:04:56.674872 9540 tablet.cc:2369] T 42a3dfb726e940f3adea14faa15f3ef4 P 286752113f2145c9a37a31e2c1dbc2e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 06:04:56.727681 9797 raft_consensus.cc:482] T 42a3dfb726e940f3adea14faa15f3ef4 P d509319a6f964980acdf6d7a77baf7ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:04:56.727877 9797 raft_consensus.cc:504] T 42a3dfb726e940f3adea14faa15f3ef4 P d509319a6f964980acdf6d7a77baf7ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "286752113f2145c9a37a31e2c1dbc2e3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 41011 } } peers { permanent_uuid: "d509319a6f964980acdf6d7a77baf7ca" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 46611 } } peers { permanent_uuid: "41a5e493e78f4c9c8f46b61e4310d618" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 38891 } }
I20260808 06:04:56.728987 9797 leader_election.cc:288] T 42a3dfb726e940f3adea14faa15f3ef4 P d509319a6f964980acdf6d7a77baf7ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 286752113f2145c9a37a31e2c1dbc2e3 (127.1.151.130:41011), 41a5e493e78f4c9c8f46b61e4310d618 (127.1.151.131:38891)
I20260808 06:04:56.730793 9781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:56.729347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:35861 (local address 127.1.151.130:41011)
0808 06:04:56.729477 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:56.729482 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:56.729526 (+ 44us) server_negotiation.cc:407] Connection header received
0808 06:04:56.729576 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:56.729579 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:56.729619 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:56.729688 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:56.730417 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:56.730421 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:56.730424 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:04:56.730437 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:56.730457 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:04:56.730465 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:56.730543 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:56.730631 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 06:04:56.730669 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 06:04:56.730793 9325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:56.729232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:41011 (local address 127.1.151.129:35861)
0808 06:04:56.729362 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:56.729378 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:56.729458 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:56.729733 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:56.729736 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:56.729754 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:56.730242 (+ 488us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 06:04:56.730244 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:56.730257 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 06:04:56.730285 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:56.730536 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:56.730539 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:56.730542 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 06:04:56.730627 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 06:04:56.730665 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 06:04:56.731133 9789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:56.729641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:43159 (local address 127.1.151.131:38891)
0808 06:04:56.729838 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:56.729843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:56.729910 (+ 67us) server_negotiation.cc:407] Connection header received
0808 06:04:56.730006 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:56.730009 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:56.730043 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:56.730108 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:56.730752 (+ 644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:56.730755 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:56.730758 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:04:56.730770 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:56.730786 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:04:56.730793 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:56.730866 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:56.730966 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 06:04:56.731009 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
I20260808 06:04:56.731508 9802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:56.729494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:38891 (local address 127.1.151.129:43159)
0808 06:04:56.729818 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:56.729831 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:56.729922 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:56.730146 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:56.730173 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:56.730187 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:04:56.730586 (+ 399us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 06:04:56.730588 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:04:56.730599 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 06:04:56.730630 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:04:56.730861 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:04:56.730863 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:04:56.730866 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 06:04:56.731330 (+ 464us) client_negotiation.cc:240] Negotiation successful
0808 06:04:56.731367 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":137,"threads_started":1}
I20260808 06:04:56.731442 9458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42a3dfb726e940f3adea14faa15f3ef4" candidate_uuid: "d509319a6f964980acdf6d7a77baf7ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "286752113f2145c9a37a31e2c1dbc2e3" is_pre_election: true
I20260808 06:04:56.731742 9458 raft_consensus.cc:2454] T 42a3dfb726e940f3adea14faa15f3ef4 P 286752113f2145c9a37a31e2c1dbc2e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d509319a6f964980acdf6d7a77baf7ca in term 0.
I20260808 06:04:56.732192 9691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42a3dfb726e940f3adea14faa15f3ef4" candidate_uuid: "d509319a6f964980acdf6d7a77baf7ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41a5e493e78f4c9c8f46b61e4310d618" is_pre_election: true
I20260808 06:04:56.732241 9148 leader_election.cc:302] T 42a3dfb726e940f3adea14faa15f3ef4 P d509319a6f964980acdf6d7a77baf7ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 286752113f2145c9a37a31e2c1dbc2e3, d509319a6f964980acdf6d7a77baf7ca; no voters:
I20260808 06:04:56.732488 9691 raft_consensus.cc:2454] T 42a3dfb726e940f3adea14faa15f3ef4 P 41a5e493e78f4c9c8f46b61e4310d618 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d509319a6f964980acdf6d7a77baf7ca in term 0.
I20260808 06:04:56.732503 9797 raft_consensus.cc:2781] T 42a3dfb726e940f3adea14faa15f3ef4 P d509319a6f964980acdf6d7a77baf7ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 06:04:56.732581 9797 raft_consensus.cc:482] T 42a3dfb726e940f3adea14faa15f3ef4 P d509319a6f964980acdf6d7a77baf7ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 06:04:56.732638 9797 raft_consensus.cc:3037] T 42a3dfb726e940f3adea14faa15f3ef4 P d509319a6f964980acdf6d7a77baf7ca [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:56.733743 9797 raft_consensus.cc:504] T 42a3dfb726e940f3adea14faa15f3ef4 P d509319a6f964980acdf6d7a77baf7ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "286752113f2145c9a37a31e2c1dbc2e3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 41011 } } peers { permanent_uuid: "d509319a6f964980acdf6d7a77baf7ca" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 46611 } } peers { permanent_uuid: "41a5e493e78f4c9c8f46b61e4310d618" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 38891 } }
I20260808 06:04:56.734046 9797 leader_election.cc:288] T 42a3dfb726e940f3adea14faa15f3ef4 P d509319a6f964980acdf6d7a77baf7ca [CANDIDATE]: Term 1 election: Requested vote from peers 286752113f2145c9a37a31e2c1dbc2e3 (127.1.151.130:41011), 41a5e493e78f4c9c8f46b61e4310d618 (127.1.151.131:38891)
I20260808 06:04:56.734406 9458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42a3dfb726e940f3adea14faa15f3ef4" candidate_uuid: "d509319a6f964980acdf6d7a77baf7ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "286752113f2145c9a37a31e2c1dbc2e3"
I20260808 06:04:56.734472 9691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42a3dfb726e940f3adea14faa15f3ef4" candidate_uuid: "d509319a6f964980acdf6d7a77baf7ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41a5e493e78f4c9c8f46b61e4310d618"
I20260808 06:04:56.734551 9458 raft_consensus.cc:3037] T 42a3dfb726e940f3adea14faa15f3ef4 P 286752113f2145c9a37a31e2c1dbc2e3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:56.734575 9691 raft_consensus.cc:3037] T 42a3dfb726e940f3adea14faa15f3ef4 P 41a5e493e78f4c9c8f46b61e4310d618 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:56.735905 9458 raft_consensus.cc:2454] T 42a3dfb726e940f3adea14faa15f3ef4 P 286752113f2145c9a37a31e2c1dbc2e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d509319a6f964980acdf6d7a77baf7ca in term 1.
I20260808 06:04:56.735905 9691 raft_consensus.cc:2454] T 42a3dfb726e940f3adea14faa15f3ef4 P 41a5e493e78f4c9c8f46b61e4310d618 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d509319a6f964980acdf6d7a77baf7ca in term 1.
I20260808 06:04:56.736217 9148 leader_election.cc:302] T 42a3dfb726e940f3adea14faa15f3ef4 P d509319a6f964980acdf6d7a77baf7ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41a5e493e78f4c9c8f46b61e4310d618, d509319a6f964980acdf6d7a77baf7ca; no voters:
I20260808 06:04:56.736380 9797 raft_consensus.cc:2781] T 42a3dfb726e940f3adea14faa15f3ef4 P d509319a6f964980acdf6d7a77baf7ca [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:04:56.736604 9797 raft_consensus.cc:686] T 42a3dfb726e940f3adea14faa15f3ef4 P d509319a6f964980acdf6d7a77baf7ca [term 1 LEADER]: Becoming Leader. State: Replica: d509319a6f964980acdf6d7a77baf7ca, State: Running, Role: LEADER
I20260808 06:04:56.736929 9797 consensus_queue.cc:237] T 42a3dfb726e940f3adea14faa15f3ef4 P d509319a6f964980acdf6d7a77baf7ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "286752113f2145c9a37a31e2c1dbc2e3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 41011 } } peers { permanent_uuid: "d509319a6f964980acdf6d7a77baf7ca" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 46611 } } peers { permanent_uuid: "41a5e493e78f4c9c8f46b61e4310d618" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 38891 } }
I20260808 06:04:56.740137 9030 catalog_manager.cc:5495] T 42a3dfb726e940f3adea14faa15f3ef4 P d509319a6f964980acdf6d7a77baf7ca reported cstate change: term changed from 0 to 1, leader changed from <none> to d509319a6f964980acdf6d7a77baf7ca (127.1.151.129). New cstate: current_term: 1 leader_uuid: "d509319a6f964980acdf6d7a77baf7ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "286752113f2145c9a37a31e2c1dbc2e3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 41011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d509319a6f964980acdf6d7a77baf7ca" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 46611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41a5e493e78f4c9c8f46b61e4310d618" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 38891 } health_report { overall_health: UNKNOWN } } }
I20260808 06:04:56.752445 9783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:56.751245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50634 (local address 127.1.151.129:46611)
0808 06:04:56.751368 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:56.751372 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:56.751385 (+ 13us) server_negotiation.cc:407] Connection header received
0808 06:04:56.751452 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:56.751454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:56.751509 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:56.751575 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:56.752152 (+ 577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:56.752155 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:56.752157 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:04:56.752169 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:56.752188 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:04:56.752196 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:56.752265 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:56.752314 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 06:04:56.752346 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 06:04:56.758216 9691 raft_consensus.cc:1261] T 42a3dfb726e940f3adea14faa15f3ef4 P 41a5e493e78f4c9c8f46b61e4310d618 [term 1 FOLLOWER]: Refusing update from remote peer d509319a6f964980acdf6d7a77baf7ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:04:56.758216 9458 raft_consensus.cc:1261] T 42a3dfb726e940f3adea14faa15f3ef4 P 286752113f2145c9a37a31e2c1dbc2e3 [term 1 FOLLOWER]: Refusing update from remote peer d509319a6f964980acdf6d7a77baf7ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260808 06:04:56.758574 9324 tablet.cc:2369] T 42a3dfb726e940f3adea14faa15f3ef4 P d509319a6f964980acdf6d7a77baf7ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 06:04:56.758946 9803 consensus_queue.cc:1035] T 42a3dfb726e940f3adea14faa15f3ef4 P d509319a6f964980acdf6d7a77baf7ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "286752113f2145c9a37a31e2c1dbc2e3" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:04:56.759119 9797 consensus_queue.cc:1035] T 42a3dfb726e940f3adea14faa15f3ef4 P d509319a6f964980acdf6d7a77baf7ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "41a5e493e78f4c9c8f46b61e4310d618" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 38891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:04:56.766098 9808 mvcc.cc:205] Tried to move back new op lower bound from 7316148620314492928 to 7316148620236922880. Current Snapshot: MvccSnapshot[applied={T|T < 7316148620314492928}]
I20260808 06:04:56.773679 9023 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45474:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:04:56.773833 9023 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45474:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:04:56.776170 9023 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2ea0b31c74a34f719fe39e8b2607abc3: Sending DeleteTablet for 3 replicas of tablet 42a3dfb726e940f3adea14faa15f3ef4
I20260808 06:04:56.777024 9658 tablet_service.cc:1514] Processing DeleteTablet for tablet 42a3dfb726e940f3adea14faa15f3ef4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:04:56 UTC) from {username='slave'} at 127.0.0.1:33122
I20260808 06:04:56.777071 9421 tablet_service.cc:1514] Processing DeleteTablet for tablet 42a3dfb726e940f3adea14faa15f3ef4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:04:56 UTC) from {username='slave'} at 127.0.0.1:49442
I20260808 06:04:56.777084 9225 tablet_service.cc:1514] Processing DeleteTablet for tablet 42a3dfb726e940f3adea14faa15f3ef4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:04:56 UTC) from {username='slave'} at 127.0.0.1:50632
I20260808 06:04:56.777556 9824 tablet_replica.cc:330] T 42a3dfb726e940f3adea14faa15f3ef4 P d509319a6f964980acdf6d7a77baf7ca: stopping tablet replica
I20260808 06:04:56.777838 9820 tablet_replica.cc:330] T 42a3dfb726e940f3adea14faa15f3ef4 P 41a5e493e78f4c9c8f46b61e4310d618: stopping tablet replica
I20260808 06:04:56.777848 9824 raft_consensus.cc:2229] T 42a3dfb726e940f3adea14faa15f3ef4 P d509319a6f964980acdf6d7a77baf7ca [term 1 LEADER]: Raft consensus shutting down.
I20260808 06:04:56.778209 9824 raft_consensus.cc:2258] T 42a3dfb726e940f3adea14faa15f3ef4 P d509319a6f964980acdf6d7a77baf7ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:04:56.778247 9820 raft_consensus.cc:2229] T 42a3dfb726e940f3adea14faa15f3ef4 P 41a5e493e78f4c9c8f46b61e4310d618 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:04:56.778344 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 9118
I20260808 06:04:56.789363 9820 raft_consensus.cc:2258] T 42a3dfb726e940f3adea14faa15f3ef4 P 41a5e493e78f4c9c8f46b61e4310d618 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:04:56.789858 9822 tablet_replica.cc:330] T 42a3dfb726e940f3adea14faa15f3ef4 P 286752113f2145c9a37a31e2c1dbc2e3: stopping tablet replica
I20260808 06:04:56.791453 9820 ts_tablet_manager.cc:1905] T 42a3dfb726e940f3adea14faa15f3ef4 P 41a5e493e78f4c9c8f46b61e4310d618: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:04:56.794238 9820 ts_tablet_manager.cc:1918] T 42a3dfb726e940f3adea14faa15f3ef4 P 41a5e493e78f4c9c8f46b61e4310d618: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:04:56.794338 9820 log.cc:1196] T 42a3dfb726e940f3adea14faa15f3ef4 P 41a5e493e78f4c9c8f46b61e4310d618: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786169070948234-1630-0/minicluster-data/ts-2/wal/wals/42a3dfb726e940f3adea14faa15f3ef4
I20260808 06:04:56.794528 9822 raft_consensus.cc:2229] T 42a3dfb726e940f3adea14faa15f3ef4 P 286752113f2145c9a37a31e2c1dbc2e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:04:56.794664 9820 ts_tablet_manager.cc:1939] T 42a3dfb726e940f3adea14faa15f3ef4 P 41a5e493e78f4c9c8f46b61e4310d618: Deleting consensus metadata
I20260808 06:04:56.794852 9822 raft_consensus.cc:2258] T 42a3dfb726e940f3adea14faa15f3ef4 P 286752113f2145c9a37a31e2c1dbc2e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:04:56.795984 9005 catalog_manager.cc:4841] TS 41a5e493e78f4c9c8f46b61e4310d618 (127.1.151.131:38891): tablet 42a3dfb726e940f3adea14faa15f3ef4 (table test-table [id=1ee5289c1dbc4eb9a3cacb96cbcfd7e9]) successfully deleted
I20260808 06:04:56.796203 9822 ts_tablet_manager.cc:1905] T 42a3dfb726e940f3adea14faa15f3ef4 P 286752113f2145c9a37a31e2c1dbc2e3: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 06:04:56.798030 9005 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.151.129:46611 (error 108)
I20260808 06:04:56.798239 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 9326
I20260808 06:04:56.804875 9782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:56.804494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:46611 (local address 127.0.0.1:50640)
0808 06:04:56.804665 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:56.804770 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.129:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 06:04:56.805436 9005 catalog_manager.cc:4584] TS d509319a6f964980acdf6d7a77baf7ca (127.1.151.129:46611): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 42a3dfb726e940f3adea14faa15f3ef4: Network error: Client connection negotiation failed: client connection to 127.1.151.129:46611: connect: Connection refused (error 111)
I20260808 06:04:56.810556 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 9542
I20260808 06:04:56.816677 9782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:56.810731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:41011 (local address 127.0.0.1:49444)
0808 06:04:56.816498 (+ 5767us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:56.816572 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.130:41011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5655}
I20260808 06:04:56.821825 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 8971
2026-08-08T06:04:56Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (985 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 06:04:56.848790 1630 mini_oidc.cc:152] Starting JWKS server
I20260808 06:04:56.850585 1630 webserver.cc:460] Webserver started at https://127.0.0.1:35779/ using document root <none> and password file <none>
I20260808 06:04:56.850739 1630 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 06:04:56.851007 1630 webserver.cc:460] Webserver started at http://127.0.0.1:36063/ using document root <none> and password file <none>
2026-08-08T06:04:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:04:56Z Disabled control of system clock
I20260808 06:04:56.860929 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:32855
--webserver_interface=127.1.151.190
--webserver_port=0
--builtin_ntp_servers=127.1.151.148:43537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:32855
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35779/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786169070948234-1630-0/testchainca.cert with env {}
W20260808 06:04:56.969404 9837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:56.969641 9837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:56.969677 9837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:56.973541 9837 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:04:56.973596 9837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:56.973610 9837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:56.973625 9837 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:04:56.973639 9837 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 06:04:56.973652 9837 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 06:04:56.973667 9837 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35779/jwks.json
I20260808 06:04:56.978884 9837 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:43537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:32855
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:32855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35779/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786169070948234-1630-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:56.979847 9837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:56.980782 9837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:56.988896 9849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:56.989015 9837 server_base.cc:1034] running on GCE node
W20260808 06:04:56.988896 9844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:56.989193 9843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:56.989758 9837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:56.990831 9837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:56.991994 9837 hybrid_clock.cc:648] HybridClock initialized: now 1786169096991980 us; error 25 us; skew 500 ppm
I20260808 06:04:56.993664 9837 webserver.cc:460] Webserver started at http://127.1.151.190:41801/ using document root <none> and password file <none>
I20260808 06:04:56.994244 9837 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:56.994318 9837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:56.994510 9837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:56.996469 9837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "4b3fefdf93d4447187635a7e6943e6e3"
format_stamp: "Formatted at 2026-08-08 06:04:56 on dist-test-slave-zj8q"
server_key: "bef448fcb9e2fb09914a99acf1ae80c9"
server_key_iv: "7d07f485eb79682718a1b7eb38e4d51f"
server_key_version: "encryptionkey@0"
I20260808 06:04:56.996987 9837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "4b3fefdf93d4447187635a7e6943e6e3"
format_stamp: "Formatted at 2026-08-08 06:04:56 on dist-test-slave-zj8q"
server_key: "bef448fcb9e2fb09914a99acf1ae80c9"
server_key_iv: "7d07f485eb79682718a1b7eb38e4d51f"
server_key_version: "encryptionkey@0"
I20260808 06:04:57.000516 9837 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260808 06:04:57.002849 9858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:57.003958 9837 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:04:57.004083 9837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "4b3fefdf93d4447187635a7e6943e6e3"
format_stamp: "Formatted at 2026-08-08 06:04:56 on dist-test-slave-zj8q"
server_key: "bef448fcb9e2fb09914a99acf1ae80c9"
server_key_iv: "7d07f485eb79682718a1b7eb38e4d51f"
server_key_version: "encryptionkey@0"
I20260808 06:04:57.004189 9837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:57.015954 9837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:57.016618 9837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:57.016788 9837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:57.023509 9837 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:32855
I20260808 06:04:57.023525 9952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:32855 every 8 connection(s)
I20260808 06:04:57.024423 9837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:04:57.026464 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 9837
I20260808 06:04:57.026577 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:04:57.026861 1630 external_mini_cluster.cc:1428] Setting key 94de62d693c8d123bb60b386db84aae3
I20260808 06:04:57.027793 9954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:57.033951 9954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b3fefdf93d4447187635a7e6943e6e3: Bootstrap starting.
I20260808 06:04:57.035681 9957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:57.028156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36248 (local address 127.1.151.190:32855)
0808 06:04:57.028743 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:57.028759 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:57.028796 (+ 37us) server_negotiation.cc:407] Connection header received
0808 06:04:57.029440 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:57.029456 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:57.029827 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:57.030088 (+ 261us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:57.030479 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:57.031423 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:57.032588 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:57.033048 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:57.033175 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:57.033977 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:57.033981 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:57.033994 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:04:57.034448 (+ 454us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:57.034503 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:04:57.034524 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:57.034595 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:57.034759 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 06:04:57.035003 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":111,"threads_started":1}
I20260808 06:04:57.036346 9954 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b3fefdf93d4447187635a7e6943e6e3: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:57.037184 9954 log.cc:824] T 00000000000000000000000000000000 P 4b3fefdf93d4447187635a7e6943e6e3: Log is configured to *not* fsync() on all Append() calls
I20260808 06:04:57.038383 1630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 06:04:57.039316 9954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b3fefdf93d4447187635a7e6943e6e3: No bootstrap required, opened a new log
I20260808 06:04:57.042761 9954 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b3fefdf93d4447187635a7e6943e6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b3fefdf93d4447187635a7e6943e6e3" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 32855 } }
I20260808 06:04:57.042964 9954 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b3fefdf93d4447187635a7e6943e6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:57.043011 9954 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b3fefdf93d4447187635a7e6943e6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b3fefdf93d4447187635a7e6943e6e3, State: Initialized, Role: FOLLOWER
I20260808 06:04:57.043370 9954 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b3fefdf93d4447187635a7e6943e6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b3fefdf93d4447187635a7e6943e6e3" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 32855 } }
I20260808 06:04:57.043491 9954 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b3fefdf93d4447187635a7e6943e6e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:04:57.043534 9954 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b3fefdf93d4447187635a7e6943e6e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:04:57.043602 9954 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b3fefdf93d4447187635a7e6943e6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:57.044338 9954 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b3fefdf93d4447187635a7e6943e6e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b3fefdf93d4447187635a7e6943e6e3" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 32855 } }
I20260808 06:04:57.044618 9954 leader_election.cc:302] T 00000000000000000000000000000000 P 4b3fefdf93d4447187635a7e6943e6e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b3fefdf93d4447187635a7e6943e6e3; no voters:
I20260808 06:04:57.044857 9954 leader_election.cc:288] T 00000000000000000000000000000000 P 4b3fefdf93d4447187635a7e6943e6e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:04:57.044966 9960 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b3fefdf93d4447187635a7e6943e6e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:04:57.045205 9960 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b3fefdf93d4447187635a7e6943e6e3 [term 1 LEADER]: Becoming Leader. State: Replica: 4b3fefdf93d4447187635a7e6943e6e3, State: Running, Role: LEADER
I20260808 06:04:57.045574 9960 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b3fefdf93d4447187635a7e6943e6e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b3fefdf93d4447187635a7e6943e6e3" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 32855 } }
I20260808 06:04:57.046005 9954 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b3fefdf93d4447187635a7e6943e6e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:04:57.047052 9962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b3fefdf93d4447187635a7e6943e6e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b3fefdf93d4447187635a7e6943e6e3. Latest consensus state: current_term: 1 leader_uuid: "4b3fefdf93d4447187635a7e6943e6e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b3fefdf93d4447187635a7e6943e6e3" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 32855 } } }
I20260808 06:04:57.047199 9962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b3fefdf93d4447187635a7e6943e6e3 [sys.catalog]: This master's current role is: LEADER
I20260808 06:04:57.047499 9961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b3fefdf93d4447187635a7e6943e6e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b3fefdf93d4447187635a7e6943e6e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b3fefdf93d4447187635a7e6943e6e3" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 32855 } } }
I20260808 06:04:57.047583 9961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b3fefdf93d4447187635a7e6943e6e3 [sys.catalog]: This master's current role is: LEADER
I20260808 06:04:57.048344 9972 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:04:57.050941 9972 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:04:57.056699 9972 catalog_manager.cc:1347] Generated new cluster ID: e111869f842d492182e3fd3f17c7b8cf
I20260808 06:04:57.056783 9972 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:04:57.064371 9972 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:04:57.065552 9972 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:04:57.074075 9972 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b3fefdf93d4447187635a7e6943e6e3: Generated new TSK 0
I20260808 06:04:57.074875 9972 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 06:04:57.106117 9997 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 06:04:57.106307 9997 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 06:04:57.106953 9957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:57.089474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36250 (local address 127.1.151.190:32855)
0808 06:04:57.089600 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:57.089605 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:57.089788 (+ 183us) server_negotiation.cc:407] Connection header received
0808 06:04:57.089981 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:57.089984 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:57.090040 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:57.090181 (+ 141us) server_negotiation.cc:226] Negotiated authn=JWT
0808 06:04:57.090412 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:57.091252 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:57.092116 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:57.092462 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:57.092561 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:57.092847 (+ 286us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 06:04:57.106504 (+ 13657us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35779/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 06:04:57.106797 (+ 293us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36250: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35779/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":44}
W20260808 06:04:57.107038 9957 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36250: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35779/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 06:04:57.108243 9996 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:04:57.089384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:32855 (local address 127.0.0.1:36250)
0808 06:04:57.089715 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:57.089727 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:57.089840 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:57.090197 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:57.090199 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:57.090208 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0808 06:04:57.090315 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:57.090320 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:57.091375 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:57.091378 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:57.092016 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:57.092025 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:57.092557 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:57.092560 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:57.092622 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:57.092681 (+ 59us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 06:04:57.106734 (+ 14053us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35779/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 06:04:57.106944 (+ 210us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.151.190:32855: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35779/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":112,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.1.151.190:32855: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35779/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 06:04:57.109120 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 9837
2026-08-08T06:04:57Z chronyd exiting
I20260808 06:04:57.155817 1630 test_util.cc:175] -----------------------------------------------
I20260808 06:04:57.155903 1630 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786169070948234-1630-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (308 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 06:04:57.157076 1630 mini_oidc.cc:152] Starting JWKS server
I20260808 06:04:57.157904 1630 webserver.cc:460] Webserver started at https://127.0.0.1:37469/ using document root <none> and password file <none>
I20260808 06:04:57.157989 1630 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 06:04:57.158247 1630 webserver.cc:460] Webserver started at http://127.0.0.1:44851/ using document root <none> and password file <none>
2026-08-08T06:04:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:04:57Z Disabled control of system clock
I20260808 06:04:57.168113 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:34887
--webserver_interface=127.1.151.190
--webserver_port=0
--builtin_ntp_servers=127.1.151.148:40769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:34887
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37469/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786169070948234-1630-0/testchainca.cert with env {}
W20260808 06:04:57.278331 10010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:57.278581 10010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:57.278620 10010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:57.282487 10010 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:04:57.282538 10010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:57.282552 10010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:57.282567 10010 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:04:57.282582 10010 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 06:04:57.282599 10010 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 06:04:57.282613 10010 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37469/jwks.json
I20260808 06:04:57.287768 10010 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:40769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:34887
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:34887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37469/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786169070948234-1630-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:57.288697 10010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:57.289616 10010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:57.297814 10023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:57.297950 10010 server_base.cc:1034] running on GCE node
W20260808 06:04:57.297802 10017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:57.297806 10018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:57.298642 10010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:57.299682 10010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:57.300858 10010 hybrid_clock.cc:648] HybridClock initialized: now 1786169097300841 us; error 25 us; skew 500 ppm
I20260808 06:04:57.302685 10010 webserver.cc:460] Webserver started at http://127.1.151.190:35431/ using document root <none> and password file <none>
I20260808 06:04:57.303272 10010 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:57.303342 10010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:57.303540 10010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:57.305629 10010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "640d4c10adc24922b65a1728c4989ca3"
format_stamp: "Formatted at 2026-08-08 06:04:57 on dist-test-slave-zj8q"
server_key: "0af874ff127752d8f0a5f3f89e59c470"
server_key_iv: "b547c7b52dc9fc0d18faf1fe7adf816e"
server_key_version: "encryptionkey@0"
I20260808 06:04:57.306188 10010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "640d4c10adc24922b65a1728c4989ca3"
format_stamp: "Formatted at 2026-08-08 06:04:57 on dist-test-slave-zj8q"
server_key: "0af874ff127752d8f0a5f3f89e59c470"
server_key_iv: "b547c7b52dc9fc0d18faf1fe7adf816e"
server_key_version: "encryptionkey@0"
I20260808 06:04:57.309597 10010 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 06:04:57.311892 10030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:57.313036 10010 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:04:57.313195 10010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "640d4c10adc24922b65a1728c4989ca3"
format_stamp: "Formatted at 2026-08-08 06:04:57 on dist-test-slave-zj8q"
server_key: "0af874ff127752d8f0a5f3f89e59c470"
server_key_iv: "b547c7b52dc9fc0d18faf1fe7adf816e"
server_key_version: "encryptionkey@0"
I20260808 06:04:57.313294 10010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:57.327735 10010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:57.328330 10010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:57.328518 10010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:57.335310 10010 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:34887
I20260808 06:04:57.335330 10116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:34887 every 8 connection(s)
I20260808 06:04:57.336251 10010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:04:57.339248 10118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:57.343837 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 10010
I20260808 06:04:57.343968 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:04:57.344287 1630 external_mini_cluster.cc:1428] Setting key 20d25ed5385d78f2da8fd9d2b473ee5a
I20260808 06:04:57.344933 10118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 640d4c10adc24922b65a1728c4989ca3: Bootstrap starting.
I20260808 06:04:57.347344 10118 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 640d4c10adc24922b65a1728c4989ca3: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:57.348073 10118 log.cc:824] T 00000000000000000000000000000000 P 640d4c10adc24922b65a1728c4989ca3: Log is configured to *not* fsync() on all Append() calls
I20260808 06:04:57.349880 10118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 640d4c10adc24922b65a1728c4989ca3: No bootstrap required, opened a new log
I20260808 06:04:57.353315 10126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:57.345702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39814 (local address 127.1.151.190:34887)
0808 06:04:57.346418 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:57.346432 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:57.346468 (+ 36us) server_negotiation.cc:407] Connection header received
0808 06:04:57.347111 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:57.347129 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:57.347530 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:57.347831 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:57.348286 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:57.349270 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:57.350190 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:57.350658 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:57.350809 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:57.351671 (+ 862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:57.351677 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:57.351688 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:04:57.352054 (+ 366us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:57.352106 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:04:57.352126 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:57.352233 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:57.352374 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 06:04:57.352620 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":139,"threads_started":1}
I20260808 06:04:57.353437 10118 raft_consensus.cc:348] T 00000000000000000000000000000000 P 640d4c10adc24922b65a1728c4989ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "640d4c10adc24922b65a1728c4989ca3" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34887 } }
I20260808 06:04:57.353667 10118 raft_consensus.cc:374] T 00000000000000000000000000000000 P 640d4c10adc24922b65a1728c4989ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:57.353734 10118 raft_consensus.cc:729] T 00000000000000000000000000000000 P 640d4c10adc24922b65a1728c4989ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 640d4c10adc24922b65a1728c4989ca3, State: Initialized, Role: FOLLOWER
I20260808 06:04:57.354175 10118 consensus_queue.cc:260] T 00000000000000000000000000000000 P 640d4c10adc24922b65a1728c4989ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "640d4c10adc24922b65a1728c4989ca3" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34887 } }
I20260808 06:04:57.354321 10118 raft_consensus.cc:388] T 00000000000000000000000000000000 P 640d4c10adc24922b65a1728c4989ca3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:04:57.354384 10118 raft_consensus.cc:482] T 00000000000000000000000000000000 P 640d4c10adc24922b65a1728c4989ca3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:04:57.354477 10118 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 640d4c10adc24922b65a1728c4989ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:57.355345 10118 raft_consensus.cc:504] T 00000000000000000000000000000000 P 640d4c10adc24922b65a1728c4989ca3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "640d4c10adc24922b65a1728c4989ca3" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34887 } }
I20260808 06:04:57.355674 10118 leader_election.cc:302] T 00000000000000000000000000000000 P 640d4c10adc24922b65a1728c4989ca3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 640d4c10adc24922b65a1728c4989ca3; no voters:
I20260808 06:04:57.355952 10118 leader_election.cc:288] T 00000000000000000000000000000000 P 640d4c10adc24922b65a1728c4989ca3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:04:57.356026 10128 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 640d4c10adc24922b65a1728c4989ca3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:04:57.356076 1630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 06:04:57.356211 10128 raft_consensus.cc:686] T 00000000000000000000000000000000 P 640d4c10adc24922b65a1728c4989ca3 [term 1 LEADER]: Becoming Leader. State: Replica: 640d4c10adc24922b65a1728c4989ca3, State: Running, Role: LEADER
I20260808 06:04:57.356503 10128 consensus_queue.cc:237] T 00000000000000000000000000000000 P 640d4c10adc24922b65a1728c4989ca3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "640d4c10adc24922b65a1728c4989ca3" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34887 } }
I20260808 06:04:57.357196 10118 sys_catalog.cc:564] T 00000000000000000000000000000000 P 640d4c10adc24922b65a1728c4989ca3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:04:57.358079 10129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 640d4c10adc24922b65a1728c4989ca3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "640d4c10adc24922b65a1728c4989ca3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "640d4c10adc24922b65a1728c4989ca3" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34887 } } }
I20260808 06:04:57.358238 10129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 640d4c10adc24922b65a1728c4989ca3 [sys.catalog]: This master's current role is: LEADER
I20260808 06:04:57.358569 10131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 640d4c10adc24922b65a1728c4989ca3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 640d4c10adc24922b65a1728c4989ca3. Latest consensus state: current_term: 1 leader_uuid: "640d4c10adc24922b65a1728c4989ca3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "640d4c10adc24922b65a1728c4989ca3" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34887 } } }
I20260808 06:04:57.358628 10144 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:04:57.358677 10131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 640d4c10adc24922b65a1728c4989ca3 [sys.catalog]: This master's current role is: LEADER
I20260808 06:04:57.361685 10144 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:04:57.366459 10144 catalog_manager.cc:1347] Generated new cluster ID: 741bd5ded1c243aea21c3ec1da16732f
I20260808 06:04:57.366523 10144 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:04:57.379971 10144 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:04:57.380890 10144 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:04:57.393636 10144 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 640d4c10adc24922b65a1728c4989ca3: Generated new TSK 0
I20260808 06:04:57.394337 10144 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 06:04:57.424665 10173 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 06:04:57.424764 10173 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 06:04:57.425493 10126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:57.402680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39822 (local address 127.1.151.190:34887)
0808 06:04:57.402795 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:57.402800 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:57.402978 (+ 178us) server_negotiation.cc:407] Connection header received
0808 06:04:57.403138 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:57.403140 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:57.403186 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:57.403277 (+ 91us) server_negotiation.cc:226] Negotiated authn=JWT
0808 06:04:57.403511 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:57.404041 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:57.404911 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:57.405134 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:57.405220 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:57.405517 (+ 297us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 06:04:57.425030 (+ 19513us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37469/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 06:04:57.425321 (+ 291us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39822: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37469/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":37}
W20260808 06:04:57.425591 10126 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39822: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37469/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 06:04:57.425621 10172 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:04:57.402606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34887 (local address 127.0.0.1:39822)
0808 06:04:57.402899 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:57.402910 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:57.403017 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:57.403289 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:57.403293 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:57.403301 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0808 06:04:57.403413 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:57.403419 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:57.404161 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:57.404164 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:57.404801 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:57.404809 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:57.405244 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:57.405248 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:57.405341 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:57.405363 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 06:04:57.425253 (+ 19890us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37469/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 06:04:57.425434 (+ 181us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.151.190:34887: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37469/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":116,"threads_started":1}
I20260808 06:04:57.426295 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 10010
2026-08-08T06:04:57Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (320 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 06:04:57.477509 1630 mini_oidc.cc:152] Starting JWKS server
I20260808 06:04:57.478439 1630 webserver.cc:460] Webserver started at https://127.0.0.1:39189/ using document root <none> and password file <none>
I20260808 06:04:57.478547 1630 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 06:04:57.478797 1630 webserver.cc:460] Webserver started at http://127.0.0.1:34217/ using document root <none> and password file <none>
2026-08-08T06:04:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:04:57Z Disabled control of system clock
I20260808 06:04:57.488633 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:35823
--webserver_interface=127.1.151.190
--webserver_port=0
--builtin_ntp_servers=127.1.151.148:37701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:35823
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39189/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 06:04:57.598558 10184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:57.598814 10184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:57.598850 10184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:57.602684 10184 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:04:57.602727 10184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:57.602741 10184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:57.602756 10184 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:04:57.602769 10184 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 06:04:57.602782 10184 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 06:04:57.602794 10184 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39189/jwks.json
I20260808 06:04:57.607410 10184 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:37701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:35823
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:35823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39189/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:57.608359 10184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:57.609608 10184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:57.617794 10191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:57.617799 10193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:57.617846 10184 server_base.cc:1034] running on GCE node
W20260808 06:04:57.617816 10190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:57.618665 10184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:57.619697 10184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:57.620846 10184 hybrid_clock.cc:648] HybridClock initialized: now 1786169097620823 us; error 38 us; skew 500 ppm
I20260808 06:04:57.628732 10184 webserver.cc:460] Webserver started at http://127.1.151.190:37981/ using document root <none> and password file <none>
I20260808 06:04:57.629304 10184 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:57.629370 10184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:57.629561 10184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:57.631651 10184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "76de63d0f352450c9093bed242f06827"
format_stamp: "Formatted at 2026-08-08 06:04:57 on dist-test-slave-zj8q"
server_key: "1de36dbee18dcf4fc2f7eb56944b0ac7"
server_key_iv: "8c77758d591efef5ccf3f4157a370a01"
server_key_version: "encryptionkey@0"
I20260808 06:04:57.632189 10184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "76de63d0f352450c9093bed242f06827"
format_stamp: "Formatted at 2026-08-08 06:04:57 on dist-test-slave-zj8q"
server_key: "1de36dbee18dcf4fc2f7eb56944b0ac7"
server_key_iv: "8c77758d591efef5ccf3f4157a370a01"
server_key_version: "encryptionkey@0"
I20260808 06:04:57.635897 10184 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 06:04:57.638329 10205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:57.639519 10184 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:04:57.639626 10184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "76de63d0f352450c9093bed242f06827"
format_stamp: "Formatted at 2026-08-08 06:04:57 on dist-test-slave-zj8q"
server_key: "1de36dbee18dcf4fc2f7eb56944b0ac7"
server_key_iv: "8c77758d591efef5ccf3f4157a370a01"
server_key_version: "encryptionkey@0"
I20260808 06:04:57.639745 10184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:57.648157 10184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:57.648728 10184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:57.648881 10184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:57.655493 10184 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:35823
I20260808 06:04:57.655505 10295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:35823 every 8 connection(s)
I20260808 06:04:57.656425 10184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:04:57.659611 10296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:57.664357 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 10184
I20260808 06:04:57.664466 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:04:57.664721 1630 external_mini_cluster.cc:1428] Setting key 37c94794cba7e565e8ddc17cbe6120ed
I20260808 06:04:57.665158 10296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76de63d0f352450c9093bed242f06827: Bootstrap starting.
I20260808 06:04:57.667850 10296 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76de63d0f352450c9093bed242f06827: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:57.668747 10296 log.cc:824] T 00000000000000000000000000000000 P 76de63d0f352450c9093bed242f06827: Log is configured to *not* fsync() on all Append() calls
I20260808 06:04:57.670855 10296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76de63d0f352450c9093bed242f06827: No bootstrap required, opened a new log
I20260808 06:04:57.675010 10300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:57.666086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34890 (local address 127.1.151.190:35823)
0808 06:04:57.666781 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:57.666799 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:57.666841 (+ 42us) server_negotiation.cc:407] Connection header received
0808 06:04:57.667654 (+ 813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:57.667679 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:57.668275 (+ 596us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:57.668640 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:57.669106 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:57.670437 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:57.671364 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:57.672038 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:57.672208 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:57.672905 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:57.672911 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:57.672928 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:04:57.673416 (+ 488us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:57.673484 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:04:57.673510 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:57.673610 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:57.673751 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 06:04:57.674042 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":103,"threads_started":1}
I20260808 06:04:57.675055 10296 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76de63d0f352450c9093bed242f06827 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76de63d0f352450c9093bed242f06827" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35823 } }
I20260808 06:04:57.675311 10296 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76de63d0f352450c9093bed242f06827 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:57.675371 10296 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76de63d0f352450c9093bed242f06827 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76de63d0f352450c9093bed242f06827, State: Initialized, Role: FOLLOWER
I20260808 06:04:57.675756 10296 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76de63d0f352450c9093bed242f06827 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76de63d0f352450c9093bed242f06827" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35823 } }
I20260808 06:04:57.675875 10296 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76de63d0f352450c9093bed242f06827 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:04:57.675930 10296 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76de63d0f352450c9093bed242f06827 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:04:57.676017 10296 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76de63d0f352450c9093bed242f06827 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:57.676909 10296 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76de63d0f352450c9093bed242f06827 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76de63d0f352450c9093bed242f06827" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35823 } }
I20260808 06:04:57.677246 10296 leader_election.cc:302] T 00000000000000000000000000000000 P 76de63d0f352450c9093bed242f06827 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76de63d0f352450c9093bed242f06827; no voters:
I20260808 06:04:57.677539 10296 leader_election.cc:288] T 00000000000000000000000000000000 P 76de63d0f352450c9093bed242f06827 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:04:57.677677 1630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 06:04:57.677670 10302 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76de63d0f352450c9093bed242f06827 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:04:57.677881 10302 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76de63d0f352450c9093bed242f06827 [term 1 LEADER]: Becoming Leader. State: Replica: 76de63d0f352450c9093bed242f06827, State: Running, Role: LEADER
I20260808 06:04:57.678206 10302 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76de63d0f352450c9093bed242f06827 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76de63d0f352450c9093bed242f06827" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35823 } }
I20260808 06:04:57.678993 10296 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76de63d0f352450c9093bed242f06827 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:04:57.679821 10303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76de63d0f352450c9093bed242f06827 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76de63d0f352450c9093bed242f06827" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76de63d0f352450c9093bed242f06827" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35823 } } }
I20260808 06:04:57.679795 10304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76de63d0f352450c9093bed242f06827 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76de63d0f352450c9093bed242f06827. Latest consensus state: current_term: 1 leader_uuid: "76de63d0f352450c9093bed242f06827" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76de63d0f352450c9093bed242f06827" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35823 } } }
I20260808 06:04:57.679948 10303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76de63d0f352450c9093bed242f06827 [sys.catalog]: This master's current role is: LEADER
I20260808 06:04:57.679948 10304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76de63d0f352450c9093bed242f06827 [sys.catalog]: This master's current role is: LEADER
I20260808 06:04:57.680397 10319 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:04:57.683499 10319 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:04:57.689071 10319 catalog_manager.cc:1347] Generated new cluster ID: c6c83400c2df4f09ac8313def926e7fb
I20260808 06:04:57.689126 10319 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:04:57.695628 10319 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:04:57.696820 10319 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:04:57.704098 10319 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76de63d0f352450c9093bed242f06827: Generated new TSK 0
I20260808 06:04:57.704739 10319 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 06:04:57.723793 10300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:57.718872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34892 (local address 127.1.151.190:35823)
0808 06:04:57.718990 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:57.718995 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:57.719119 (+ 124us) server_negotiation.cc:407] Connection header received
0808 06:04:57.719261 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:57.719264 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:57.719322 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:57.719434 (+ 112us) server_negotiation.cc:226] Negotiated authn=JWT
0808 06:04:57.719615 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:57.720475 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:57.721290 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:57.721637 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:57.721736 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:57.721963 (+ 227us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 06:04:57.723400 (+ 1437us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39189/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 06:04:57.723632 (+ 232us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34892: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39189/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 06:04:57.723860 10300 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34892: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39189/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260808 06:04:57.725327 10339 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:04:57.718768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35823 (local address 127.0.0.1:34892)
0808 06:04:57.719038 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:57.719051 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:57.719148 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:57.719423 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:57.719425 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:57.719432 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0808 06:04:57.719528 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:57.719533 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:57.720567 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:57.720569 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:57.721193 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:57.721201 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:57.721731 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:57.721734 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:57.721792 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:57.721814 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 06:04:57.723575 (+ 1761us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39189/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 06:04:57.725159 (+ 1584us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.151.190:35823: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39189/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":1396,"thread_start_us":95,"threads_started":1}
I20260808 06:04:57.725997 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 10184
E20260808 06:04:57.737061 10342 webserver.cc:533] Webserver: sslize failed:
E20260808 06:04:57.737138 10342 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-08T06:04:57Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (292 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T06:04:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:04:57Z Disabled control of system clock
I20260808 06:04:57.779394 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:36817
--webserver_interface=127.1.151.190
--webserver_port=0
--builtin_ntp_servers=127.1.151.148:36741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:36817
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786169070948234-1630-0/testchainca.cert with env {}
W20260808 06:04:57.889972 10350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:04:57.890218 10350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:04:57.890262 10350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:04:57.894186 10350 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:04:57.894235 10350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:04:57.894250 10350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:04:57.894265 10350 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:04:57.894281 10350 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 06:04:57.894309 10350 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 06:04:57.894332 10350 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 06:04:57.899576 10350 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:36741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:36817
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:36817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786169070948234-1630-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:04:57.900599 10350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:04:57.901600 10350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:04:57.909999 10364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:57.909999 10360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:04:57.910002 10359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:04:57.910395 10350 server_base.cc:1034] running on GCE node
I20260808 06:04:57.910936 10350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:04:57.911979 10350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:04:57.913147 10350 hybrid_clock.cc:648] HybridClock initialized: now 1786169097913135 us; error 31 us; skew 500 ppm
I20260808 06:04:57.914898 10350 webserver.cc:460] Webserver started at http://127.1.151.190:40553/ using document root <none> and password file <none>
I20260808 06:04:57.915449 10350 fs_manager.cc:359] Metadata directory not provided
I20260808 06:04:57.915518 10350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:04:57.915715 10350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:04:57.917630 10350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "23b5ff8e8065425d8acb009c1de4e431"
format_stamp: "Formatted at 2026-08-08 06:04:57 on dist-test-slave-zj8q"
server_key: "53ef58528398ec5e35c07666a55e951a"
server_key_iv: "e5306e1c976c1344fbf6f692d01aec7e"
server_key_version: "encryptionkey@0"
I20260808 06:04:57.918202 10350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "23b5ff8e8065425d8acb009c1de4e431"
format_stamp: "Formatted at 2026-08-08 06:04:57 on dist-test-slave-zj8q"
server_key: "53ef58528398ec5e35c07666a55e951a"
server_key_iv: "e5306e1c976c1344fbf6f692d01aec7e"
server_key_version: "encryptionkey@0"
I20260808 06:04:57.921559 10350 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 06:04:57.923777 10374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:04:57.924894 10350 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 06:04:57.925005 10350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "23b5ff8e8065425d8acb009c1de4e431"
format_stamp: "Formatted at 2026-08-08 06:04:57 on dist-test-slave-zj8q"
server_key: "53ef58528398ec5e35c07666a55e951a"
server_key_iv: "e5306e1c976c1344fbf6f692d01aec7e"
server_key_version: "encryptionkey@0"
I20260808 06:04:57.925104 10350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:04:57.936017 10350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:04:57.936671 10350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:04:57.936839 10350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:04:57.943430 10350 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:36817
I20260808 06:04:57.943446 10448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:36817 every 8 connection(s)
I20260808 06:04:57.944327 10350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:04:57.944754 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 10350
I20260808 06:04:57.944877 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:04:57.945188 1630 external_mini_cluster.cc:1428] Setting key 79c57278a9b2c6741fea5c4c8f74bf30
I20260808 06:04:57.947712 10452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:04:57.952955 10452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23b5ff8e8065425d8acb009c1de4e431: Bootstrap starting.
I20260808 06:04:57.953963 10456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:57.946588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48408 (local address 127.1.151.190:36817)
0808 06:04:57.947241 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:57.947256 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:57.947291 (+ 35us) server_negotiation.cc:407] Connection header received
0808 06:04:57.947957 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:57.947975 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:57.948358 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:57.948670 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:04:57.949075 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:57.950024 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:57.950924 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:57.951383 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:57.951538 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:57.952329 (+ 791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:04:57.952334 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:04:57.952346 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:04:57.952719 (+ 373us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:04:57.952776 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:04:57.952796 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:04:57.952881 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 06:04:57.953012 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 06:04:57.953282 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":146,"threads_started":1}
I20260808 06:04:57.955274 10452 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23b5ff8e8065425d8acb009c1de4e431: Neither blocks nor log segments found. Creating new log.
I20260808 06:04:57.956036 10452 log.cc:824] T 00000000000000000000000000000000 P 23b5ff8e8065425d8acb009c1de4e431: Log is configured to *not* fsync() on all Append() calls
I20260808 06:04:57.956660 1630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 06:04:57.957998 10452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23b5ff8e8065425d8acb009c1de4e431: No bootstrap required, opened a new log
I20260808 06:04:57.961606 10452 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23b5ff8e8065425d8acb009c1de4e431 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23b5ff8e8065425d8acb009c1de4e431" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 36817 } }
I20260808 06:04:57.961820 10452 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23b5ff8e8065425d8acb009c1de4e431 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:04:57.961879 10452 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23b5ff8e8065425d8acb009c1de4e431 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23b5ff8e8065425d8acb009c1de4e431, State: Initialized, Role: FOLLOWER
I20260808 06:04:57.962284 10452 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23b5ff8e8065425d8acb009c1de4e431 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23b5ff8e8065425d8acb009c1de4e431" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 36817 } }
I20260808 06:04:57.962407 10452 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23b5ff8e8065425d8acb009c1de4e431 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:04:57.962461 10452 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23b5ff8e8065425d8acb009c1de4e431 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:04:57.962550 10452 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23b5ff8e8065425d8acb009c1de4e431 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:04:57.963343 10452 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23b5ff8e8065425d8acb009c1de4e431 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23b5ff8e8065425d8acb009c1de4e431" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 36817 } }
I20260808 06:04:57.963650 10452 leader_election.cc:302] T 00000000000000000000000000000000 P 23b5ff8e8065425d8acb009c1de4e431 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23b5ff8e8065425d8acb009c1de4e431; no voters:
I20260808 06:04:57.963907 10452 leader_election.cc:288] T 00000000000000000000000000000000 P 23b5ff8e8065425d8acb009c1de4e431 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:04:57.964003 10460 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23b5ff8e8065425d8acb009c1de4e431 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:04:57.964210 10460 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23b5ff8e8065425d8acb009c1de4e431 [term 1 LEADER]: Becoming Leader. State: Replica: 23b5ff8e8065425d8acb009c1de4e431, State: Running, Role: LEADER
I20260808 06:04:57.964496 10460 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23b5ff8e8065425d8acb009c1de4e431 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23b5ff8e8065425d8acb009c1de4e431" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 36817 } }
I20260808 06:04:57.965054 10452 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23b5ff8e8065425d8acb009c1de4e431 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:04:57.966023 10461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23b5ff8e8065425d8acb009c1de4e431 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23b5ff8e8065425d8acb009c1de4e431" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23b5ff8e8065425d8acb009c1de4e431" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 36817 } } }
I20260808 06:04:57.966204 10461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23b5ff8e8065425d8acb009c1de4e431 [sys.catalog]: This master's current role is: LEADER
I20260808 06:04:57.966250 10462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23b5ff8e8065425d8acb009c1de4e431 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23b5ff8e8065425d8acb009c1de4e431. Latest consensus state: current_term: 1 leader_uuid: "23b5ff8e8065425d8acb009c1de4e431" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23b5ff8e8065425d8acb009c1de4e431" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 36817 } } }
I20260808 06:04:57.966343 10462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23b5ff8e8065425d8acb009c1de4e431 [sys.catalog]: This master's current role is: LEADER
I20260808 06:04:57.967209 10470 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:04:57.969844 10470 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:04:57.975481 10470 catalog_manager.cc:1347] Generated new cluster ID: 6aa4556f56e64a568eefa6129483cd57
I20260808 06:04:57.975545 10470 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:04:57.997066 10470 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:04:57.998303 10470 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:04:58.010855 10470 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23b5ff8e8065425d8acb009c1de4e431: Generated new TSK 0
I20260808 06:04:58.011683 10470 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 06:04:58.863648 10456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:04:58.033111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48420 (local address 127.1.151.190:36817)
0808 06:04:58.033254 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 06:04:58.033260 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 06:04:58.033428 (+ 168us) server_negotiation.cc:407] Connection header received
0808 06:04:58.033607 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:04:58.033610 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:04:58.033663 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:04:58.033752 (+ 89us) server_negotiation.cc:226] Negotiated authn=JWT
0808 06:04:58.033999 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:58.034560 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:58.035779 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:04:58.035991 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:04:58.036055 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:58.036361 (+ 306us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 06:04:58.863226 (+826865us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 06:04:58.863508 (+ 282us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48420: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":55}
W20260808 06:04:58.863736 10456 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48420: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 06:04:58.863865 10493 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:04:58.033011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:36817 (local address 127.0.0.1:48420)
0808 06:04:58.033343 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0808 06:04:58.033358 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 06:04:58.033477 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:04:58.033765 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:04:58.033768 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:04:58.033777 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0808 06:04:58.033911 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:58.033917 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:58.034681 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:58.034684 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:58.035671 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:04:58.035682 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:04:58.036083 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:04:58.036086 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:04:58.036183 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:04:58.036211 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 06:04:58.863486 (+827275us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 06:04:58.863682 (+ 196us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.151.190:36817: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":109,"threads_started":1}
I20260808 06:04:58.864475 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 10350
2026-08-08T06:04:58Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1119 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (129 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (126 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:04:59 dist-test-slave-zj8q krb5kdc[10516](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:04:59 dist-test-slave-zj8q krb5kdc[10516](info): set up 2 sockets
Aug 08 06:04:59 dist-test-slave-zj8q krb5kdc[10516](info): commencing operation
krb5kdc: starting...
W20260808 06:05:00.275795 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:05:00 dist-test-slave-zj8q krb5kdc[10516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:05:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:05:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.151.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
I20260808 06:05:00.391319 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:40933
--webserver_interface=127.1.151.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:46583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:40933
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:05:00.500938 10538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:00.501175 10538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:00.501211 10538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:00.505118 10538 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:05:00.505173 10538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:00.505190 10538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:00.505205 10538 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:05:00.505218 10538 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:05:00.510376 10538 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:46583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:40933
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:40933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:05:00.511379 10538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:00.512359 10538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:00.520625 10544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:00.520680 10545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:00.520648 10538 server_base.cc:1034] running on GCE node
W20260808 06:05:00.520692 10548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:00.521418 10538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:00.522512 10538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:00.523685 10538 hybrid_clock.cc:648] HybridClock initialized: now 1786169100523669 us; error 26 us; skew 500 ppm
Aug 08 06:05:00 dist-test-slave-zj8q krb5kdc[10516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169100, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:00.526336 10538 init.cc:377] Logged in from keytab as kudu/127.1.151.190@KRBTEST.COM (short username kudu)
I20260808 06:05:00.527134 10538 webserver.cc:460] Webserver started at http://127.1.151.190:34251/ using document root <none> and password file <none>
I20260808 06:05:00.527652 10538 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:00.527719 10538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:00.527921 10538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:05:00.529878 10538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "933d037d11b54ba9934bcd6a19292d0d"
format_stamp: "Formatted at 2026-08-08 06:05:00 on dist-test-slave-zj8q"
server_key: "0375c0393584bdd375b4cacc5ecfdce0"
server_key_iv: "f49ce4a2ca8cbaed8f28f0e6b44437e2"
server_key_version: "encryptionkey@0"
I20260808 06:05:00.530441 10538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "933d037d11b54ba9934bcd6a19292d0d"
format_stamp: "Formatted at 2026-08-08 06:05:00 on dist-test-slave-zj8q"
server_key: "0375c0393584bdd375b4cacc5ecfdce0"
server_key_iv: "f49ce4a2ca8cbaed8f28f0e6b44437e2"
server_key_version: "encryptionkey@0"
I20260808 06:05:00.533972 10538 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 06:05:00.536311 10561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:00.537470 10538 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:05:00.537582 10538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "933d037d11b54ba9934bcd6a19292d0d"
format_stamp: "Formatted at 2026-08-08 06:05:00 on dist-test-slave-zj8q"
server_key: "0375c0393584bdd375b4cacc5ecfdce0"
server_key_iv: "f49ce4a2ca8cbaed8f28f0e6b44437e2"
server_key_version: "encryptionkey@0"
I20260808 06:05:00.537693 10538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:00.549980 10538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:00.552574 10538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:05:00.552750 10538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:00.559401 10538 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:40933
I20260808 06:05:00.559417 10650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:40933 every 8 connection(s)
I20260808 06:05:00.560376 10538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:05:00.563067 10652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:05:00.567603 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 10538
I20260808 06:05:00.567714 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:05:00.567950 1630 external_mini_cluster.cc:1428] Setting key 295fea131fae97f95f9ee0e674e5f6ca
I20260808 06:05:00.568723 10652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 933d037d11b54ba9934bcd6a19292d0d: Bootstrap starting.
I20260808 06:05:00.571363 10652 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 933d037d11b54ba9934bcd6a19292d0d: Neither blocks nor log segments found. Creating new log.
I20260808 06:05:00.572269 10652 log.cc:824] T 00000000000000000000000000000000 P 933d037d11b54ba9934bcd6a19292d0d: Log is configured to *not* fsync() on all Append() calls
I20260808 06:05:00.574384 10652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 933d037d11b54ba9934bcd6a19292d0d: No bootstrap required, opened a new log
Aug 08 06:05:00 dist-test-slave-zj8q krb5kdc[10516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:05:00.578413 10652 raft_consensus.cc:348] T 00000000000000000000000000000000 P 933d037d11b54ba9934bcd6a19292d0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "933d037d11b54ba9934bcd6a19292d0d" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 40933 } }
I20260808 06:05:00.578704 10652 raft_consensus.cc:374] T 00000000000000000000000000000000 P 933d037d11b54ba9934bcd6a19292d0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:05:00.578796 10652 raft_consensus.cc:729] T 00000000000000000000000000000000 P 933d037d11b54ba9934bcd6a19292d0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 933d037d11b54ba9934bcd6a19292d0d, State: Initialized, Role: FOLLOWER
I20260808 06:05:00.579295 10652 consensus_queue.cc:260] T 00000000000000000000000000000000 P 933d037d11b54ba9934bcd6a19292d0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "933d037d11b54ba9934bcd6a19292d0d" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 40933 } }
I20260808 06:05:00.579463 10652 raft_consensus.cc:388] T 00000000000000000000000000000000 P 933d037d11b54ba9934bcd6a19292d0d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:05:00.579530 10652 raft_consensus.cc:482] T 00000000000000000000000000000000 P 933d037d11b54ba9934bcd6a19292d0d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:05:00.579628 10652 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 933d037d11b54ba9934bcd6a19292d0d [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:05:00.580717 10652 raft_consensus.cc:504] T 00000000000000000000000000000000 P 933d037d11b54ba9934bcd6a19292d0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "933d037d11b54ba9934bcd6a19292d0d" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 40933 } }
I20260808 06:05:00.581094 10652 leader_election.cc:302] T 00000000000000000000000000000000 P 933d037d11b54ba9934bcd6a19292d0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 933d037d11b54ba9934bcd6a19292d0d; no voters:
I20260808 06:05:00.581434 10652 leader_election.cc:288] T 00000000000000000000000000000000 P 933d037d11b54ba9934bcd6a19292d0d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:05:00.581534 10661 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 933d037d11b54ba9934bcd6a19292d0d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:05:00.581800 10661 raft_consensus.cc:686] T 00000000000000000000000000000000 P 933d037d11b54ba9934bcd6a19292d0d [term 1 LEADER]: Becoming Leader. State: Replica: 933d037d11b54ba9934bcd6a19292d0d, State: Running, Role: LEADER
I20260808 06:05:00.582178 10661 consensus_queue.cc:237] T 00000000000000000000000000000000 P 933d037d11b54ba9934bcd6a19292d0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "933d037d11b54ba9934bcd6a19292d0d" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 40933 } }
I20260808 06:05:00.582830 10652 sys_catalog.cc:564] T 00000000000000000000000000000000 P 933d037d11b54ba9934bcd6a19292d0d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:05:00.582871 10656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:00.569396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35350 (local address 127.1.151.190:40933)
0808 06:05:00.569828 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:00.569848 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:00.569892 (+ 44us) server_negotiation.cc:407] Connection header received
0808 06:05:00.570734 (+ 842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:00.570758 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:00.571156 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:00.571524 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:00.572217 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:00.573526 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:00.574503 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:00.575175 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:00.575335 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:00.577597 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:00.577604 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:00.577622 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:00.577653 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:00.579567 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:00.580005 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:00.580011 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:00.580017 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:00.580155 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:00.580419 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:00.580423 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:00.580424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:00.580948 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:00.581151 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:00.581413 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 06:05:00.581887 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":104,"threads_started":1}
I20260808 06:05:00.583283 10661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 933d037d11b54ba9934bcd6a19292d0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 933d037d11b54ba9934bcd6a19292d0d. Latest consensus state: current_term: 1 leader_uuid: "933d037d11b54ba9934bcd6a19292d0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "933d037d11b54ba9934bcd6a19292d0d" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 40933 } } }
I20260808 06:05:00.583398 10661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 933d037d11b54ba9934bcd6a19292d0d [sys.catalog]: This master's current role is: LEADER
I20260808 06:05:00.583827 10662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 933d037d11b54ba9934bcd6a19292d0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "933d037d11b54ba9934bcd6a19292d0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "933d037d11b54ba9934bcd6a19292d0d" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 40933 } } }
I20260808 06:05:00.583971 10662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 933d037d11b54ba9934bcd6a19292d0d [sys.catalog]: This master's current role is: LEADER
I20260808 06:05:00.585093 10665 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:05:00.587769 10665 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:05:00.592983 10665 catalog_manager.cc:1347] Generated new cluster ID: 581492ad459a4ba39419de831e6d3034
I20260808 06:05:00.593050 10665 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:05:00.612224 10665 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:05:00.613396 10665 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:05:00.624125 10665 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 933d037d11b54ba9934bcd6a19292d0d: Generated new TSK 0
I20260808 06:05:00.624856 10665 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.151.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab.
Entry for principal kudu/127.1.151.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab.
I20260808 06:05:00.659394 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.129:0
--local_ip_for_outbound_sockets=127.1.151.129
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:40933
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:46583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:05:00.769459 10696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:00.769734 10696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:00.769829 10696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:00.773936 10696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:00.774004 10696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:00.774086 10696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.129
I20260808 06:05:00.779274 10696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:46583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:40933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.129
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:05:00.780287 10696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:00.781293 10696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:00.790953 10704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:00.790962 10703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:00.790966 10706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:00.791199 10696 server_base.cc:1034] running on GCE node
I20260808 06:05:00.791615 10696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:00.792223 10696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:00.793421 10696 hybrid_clock.cc:648] HybridClock initialized: now 1786169100793405 us; error 31 us; skew 500 ppm
Aug 08 06:05:00 dist-test-slave-zj8q krb5kdc[10516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169100, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:00.796200 10696 init.cc:377] Logged in from keytab as kudu/127.1.151.129@KRBTEST.COM (short username kudu)
I20260808 06:05:00.797075 10696 webserver.cc:460] Webserver started at http://127.1.151.129:35127/ using document root <none> and password file <none>
I20260808 06:05:00.797608 10696 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:00.797676 10696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:00.797874 10696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:05:00.799887 10696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-0/data/instance:
uuid: "2d280538ff7b4ae7a563f6a1d7db9cf5"
format_stamp: "Formatted at 2026-08-08 06:05:00 on dist-test-slave-zj8q"
server_key: "49f9e8ac9d2f1886d5c9769acd80b9ce"
server_key_iv: "23d78e8a4591fd6a3b0750813c2b8d0b"
server_key_version: "encryptionkey@0"
I20260808 06:05:00.800410 10696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance:
uuid: "2d280538ff7b4ae7a563f6a1d7db9cf5"
format_stamp: "Formatted at 2026-08-08 06:05:00 on dist-test-slave-zj8q"
server_key: "49f9e8ac9d2f1886d5c9769acd80b9ce"
server_key_iv: "23d78e8a4591fd6a3b0750813c2b8d0b"
server_key_version: "encryptionkey@0"
I20260808 06:05:00.803951 10696 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 06:05:00.806243 10720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:00.807361 10696 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:05:00.807478 10696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-0/wal
uuid: "2d280538ff7b4ae7a563f6a1d7db9cf5"
format_stamp: "Formatted at 2026-08-08 06:05:00 on dist-test-slave-zj8q"
server_key: "49f9e8ac9d2f1886d5c9769acd80b9ce"
server_key_iv: "23d78e8a4591fd6a3b0750813c2b8d0b"
server_key_version: "encryptionkey@0"
I20260808 06:05:00.807611 10696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:00.840139 10696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:00.843216 10696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:05:00.843391 10696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:00.843966 10696 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:05:00.844786 10696 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:05:00.844834 10696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:00.844877 10696 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:05:00.844905 10696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:00.853035 10696 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.129:36307
I20260808 06:05:00.853091 10885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.129:36307 every 8 connection(s)
I20260808 06:05:00.853888 10696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
I20260808 06:05:00.855876 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 10696
I20260808 06:05:00.855990 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance
I20260808 06:05:00.856268 1630 external_mini_cluster.cc:1428] Setting key 63d3c286b70532acffe35cb0e7aa93e4
Aug 08 06:05:00 dist-test-slave-zj8q krb5kdc[10516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169100, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.151.130@KRBTEST.COM; defaulting to no policy
I20260808 06:05:00.867287 10656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:00.855730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:47489 (local address 127.1.151.190:40933)
0808 06:05:00.855884 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:00.855889 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:00.856784 (+ 895us) server_negotiation.cc:407] Connection header received
0808 06:05:00.858226 (+ 1442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:00.858229 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:00.858280 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:00.858355 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:00.859477 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:00.860003 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:00.861111 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:00.861420 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:00.861522 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:00.863825 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:00.863829 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:00.863832 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:00.863862 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:00.865329 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:00.865829 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:00.865833 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:00.865835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:00.865924 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:00.866238 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:00.866241 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:00.866243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:00.866471 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:00.866577 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:00.867032 (+ 455us) server_negotiation.cc:299] Negotiation successful
0808 06:05:00.867158 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 06:05:00.868048 10888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:00.856168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:40933 (local address 127.1.151.129:47489)
0808 06:05:00.856639 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:00.856691 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:00.857996 (+ 1305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:00.858513 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:00.858520 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:00.858859 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:00.859336 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:00.859349 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:00.860127 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:00.860130 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:00.860992 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:00.861002 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:00.861563 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:00.861566 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:00.861721 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:00.862263 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:00.862280 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:00.863664 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:00.865453 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:00.865458 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:00.865468 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:00.865732 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:00.866009 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:00.866011 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:00.866012 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:00.866132 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:00.866565 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:00.866570 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:00.866792 (+ 222us) client_negotiation.cc:769] Sending connection context
0808 06:05:00.866960 (+ 168us) client_negotiation.cc:240] Negotiation successful
0808 06:05:00.867228 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":127,"threads_started":1}
I20260808 06:05:00.869277 10886 heartbeater.cc:347] Connected to a master server at 127.1.151.190:40933
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.130@KRBTEST.COM" created.
I20260808 06:05:00.869544 10886 heartbeater.cc:464] Registering TS with master...
I20260808 06:05:00.870048 10886 heartbeater.cc:511] Master 127.1.151.190:40933 requested a full tablet report, sending...
I20260808 06:05:00.871713 10599 ts_manager.cc:194] Registered new tserver with Master: 2d280538ff7b4ae7a563f6a1d7db9cf5 (127.1.151.129:36307)
I20260808 06:05:00.872824 10599 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.129@KRBTEST.COM'} at 127.1.151.129:47489
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab.
Entry for principal kudu/127.1.151.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab.
I20260808 06:05:00.882674 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.130:0
--local_ip_for_outbound_sockets=127.1.151.130
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:40933
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:46583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:05:00.992421 10894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:00.992667 10894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:00.992726 10894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:00.996676 10894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:00.996730 10894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:00.996804 10894 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.130
I20260808 06:05:01.001883 10894 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:46583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:40933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.130
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:05:01.002871 10894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:01.003825 10894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:01.013057 10906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:01.013057 10902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:01.013064 10900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:01.013262 10894 server_base.cc:1034] running on GCE node
I20260808 06:05:01.013721 10894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:01.014365 10894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:01.015529 10894 hybrid_clock.cc:648] HybridClock initialized: now 1786169101015498 us; error 42 us; skew 500 ppm
Aug 08 06:05:01 dist-test-slave-zj8q krb5kdc[10516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169101, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:01.018216 10894 init.cc:377] Logged in from keytab as kudu/127.1.151.130@KRBTEST.COM (short username kudu)
I20260808 06:05:01.019006 10894 webserver.cc:460] Webserver started at http://127.1.151.130:42751/ using document root <none> and password file <none>
I20260808 06:05:01.019515 10894 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:01.019572 10894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:01.019733 10894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:05:01.021680 10894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-1/data/instance:
uuid: "a21c1ec8569f4d538839684d3827e5fb"
format_stamp: "Formatted at 2026-08-08 06:05:01 on dist-test-slave-zj8q"
server_key: "24889b763060892d32d8601f8c5e9326"
server_key_iv: "fb5a12633c333a015afa2df879d791fc"
server_key_version: "encryptionkey@0"
I20260808 06:05:01.022234 10894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance:
uuid: "a21c1ec8569f4d538839684d3827e5fb"
format_stamp: "Formatted at 2026-08-08 06:05:01 on dist-test-slave-zj8q"
server_key: "24889b763060892d32d8601f8c5e9326"
server_key_iv: "fb5a12633c333a015afa2df879d791fc"
server_key_version: "encryptionkey@0"
I20260808 06:05:01.025683 10894 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 06:05:01.028016 10923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:01.029138 10894 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:05:01.029233 10894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-1/wal
uuid: "a21c1ec8569f4d538839684d3827e5fb"
format_stamp: "Formatted at 2026-08-08 06:05:01 on dist-test-slave-zj8q"
server_key: "24889b763060892d32d8601f8c5e9326"
server_key_iv: "fb5a12633c333a015afa2df879d791fc"
server_key_version: "encryptionkey@0"
I20260808 06:05:01.029314 10894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:01.039515 10894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:01.042757 10894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:05:01.042902 10894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:01.043426 10894 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:05:01.044219 10894 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:05:01.044289 10894 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:01.044334 10894 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:05:01.044349 10894 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:01.052709 10894 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.130:33349
I20260808 06:05:01.052788 11090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.130:33349 every 8 connection(s)
I20260808 06:05:01.053601 10894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
I20260808 06:05:01.058668 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 10894
I20260808 06:05:01.058789 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance
I20260808 06:05:01.059072 1630 external_mini_cluster.cc:1428] Setting key 0ea2b15c1a4aa30718f24a35a674b90c
Aug 08 06:05:01 dist-test-slave-zj8q krb5kdc[10516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169101, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:05:01.066645 10656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:01.055432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:45517 (local address 127.1.151.190:40933)
0808 06:05:01.055552 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:01.055556 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:01.056374 (+ 818us) server_negotiation.cc:407] Connection header received
0808 06:05:01.057758 (+ 1384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:01.057762 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:01.057821 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:01.057890 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:01.059145 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:01.059715 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:01.060900 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:01.061122 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:01.061208 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:01.063515 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:01.063518 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:01.063520 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:01.063543 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:01.064728 (+ 1185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:01.065178 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:01.065182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:01.065183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:01.065259 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:01.065576 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:01.065579 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:01.065580 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:01.065757 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:01.065840 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:01.066351 (+ 511us) server_negotiation.cc:299] Negotiation successful
0808 06:05:01.066482 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 06:05:01.067390 11096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:01.055777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:40933 (local address 127.1.151.130:45517)
0808 06:05:01.056223 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:01.056265 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:01.057537 (+ 1272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:01.058074 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:01.058081 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:01.058471 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:01.058990 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:01.059006 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:01.059822 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:01.059826 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:01.060785 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:01.060796 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:01.061257 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:01.061261 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:01.061420 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:01.061965 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:01.061980 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:01.063359 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:01.064832 (+ 1473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:01.064838 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:01.064852 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:01.065082 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:01.065340 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:01.065343 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:01.065345 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:01.065473 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:01.065857 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:01.065863 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:01.066089 (+ 226us) client_negotiation.cc:769] Sending connection context
0808 06:05:01.066281 (+ 192us) client_negotiation.cc:240] Negotiation successful
0808 06:05:01.066582 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":114,"threads_started":1}
I20260808 06:05:01.068490 11093 heartbeater.cc:347] Connected to a master server at 127.1.151.190:40933
I20260808 06:05:01.068751 11093 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.1.151.131@KRBTEST.COM; defaulting to no policy
I20260808 06:05:01.069330 11093 heartbeater.cc:511] Master 127.1.151.190:40933 requested a full tablet report, sending...
I20260808 06:05:01.070382 10599 ts_manager.cc:194] Registered new tserver with Master: a21c1ec8569f4d538839684d3827e5fb (127.1.151.130:33349)
I20260808 06:05:01.071003 10599 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.130@KRBTEST.COM'} at 127.1.151.130:45517
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab.
Entry for principal kudu/127.1.151.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab.
I20260808 06:05:01.084864 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.131:0
--local_ip_for_outbound_sockets=127.1.151.131
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:40933
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:46583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:05:01.197278 11101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:01.197566 11101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:01.197616 11101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:01.201620 11101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:01.201675 11101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:01.201745 11101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.131
I20260808 06:05:01.207023 11101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:46583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:40933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.131
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:05:01.208011 11101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:01.209050 11101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:01.218442 11108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:01.218463 11109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:01.218462 11111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:01.218747 11101 server_base.cc:1034] running on GCE node
I20260808 06:05:01.219105 11101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:01.219710 11101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:01.220885 11101 hybrid_clock.cc:648] HybridClock initialized: now 1786169101220865 us; error 33 us; skew 500 ppm
Aug 08 06:05:01 dist-test-slave-zj8q krb5kdc[10516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169101, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:01.223716 11101 init.cc:377] Logged in from keytab as kudu/127.1.151.131@KRBTEST.COM (short username kudu)
I20260808 06:05:01.224558 11101 webserver.cc:460] Webserver started at http://127.1.151.131:46605/ using document root <none> and password file <none>
I20260808 06:05:01.225081 11101 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:01.225148 11101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:01.225355 11101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:05:01.227366 11101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-2/data/instance:
uuid: "a71c9a2f7f8144c2ab708992233ac1dc"
format_stamp: "Formatted at 2026-08-08 06:05:01 on dist-test-slave-zj8q"
server_key: "572b52152f76035e3c1679ab25985810"
server_key_iv: "cd0c6b04e5ab4fa117aa4dc162682ceb"
server_key_version: "encryptionkey@0"
I20260808 06:05:01.227905 11101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance:
uuid: "a71c9a2f7f8144c2ab708992233ac1dc"
format_stamp: "Formatted at 2026-08-08 06:05:01 on dist-test-slave-zj8q"
server_key: "572b52152f76035e3c1679ab25985810"
server_key_iv: "cd0c6b04e5ab4fa117aa4dc162682ceb"
server_key_version: "encryptionkey@0"
I20260808 06:05:01.231449 11101 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 06:05:01.233808 11124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:01.235013 11101 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:05:01.235136 11101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-2/wal
uuid: "a71c9a2f7f8144c2ab708992233ac1dc"
format_stamp: "Formatted at 2026-08-08 06:05:01 on dist-test-slave-zj8q"
server_key: "572b52152f76035e3c1679ab25985810"
server_key_iv: "cd0c6b04e5ab4fa117aa4dc162682ceb"
server_key_version: "encryptionkey@0"
I20260808 06:05:01.235224 11101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:01.247686 11101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:01.250630 11101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:05:01.250797 11101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:01.251395 11101 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:05:01.252447 11101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:05:01.252509 11101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:01.252578 11101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:05:01.252643 11101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:01.261659 11101 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.131:35965
I20260808 06:05:01.261693 11292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.131:35965 every 8 connection(s)
I20260808 06:05:01.262516 11101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
I20260808 06:05:01.271173 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 11101
I20260808 06:05:01.271299 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance
I20260808 06:05:01.271564 1630 external_mini_cluster.cc:1428] Setting key 7d01783f055c2974163c53810fb2723a
Aug 08 06:05:01 dist-test-slave-zj8q krb5kdc[10516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169101, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:05:01.275461 10656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:01.264244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:38675 (local address 127.1.151.190:40933)
0808 06:05:01.264367 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:01.264370 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:01.265183 (+ 813us) server_negotiation.cc:407] Connection header received
0808 06:05:01.266550 (+ 1367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:01.266554 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:01.266621 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:01.266691 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:01.267795 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:01.268320 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:01.269425 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:01.269650 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:01.269739 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:01.272212 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:01.272215 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:01.272217 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:01.272242 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:01.273546 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:01.274016 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:01.274019 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:01.274020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:01.274097 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:01.274429 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:01.274432 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:01.274434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:01.274620 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:01.274706 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:01.275187 (+ 481us) server_negotiation.cc:299] Negotiation successful
0808 06:05:01.275305 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 06:05:01.276199 11295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:01.264585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:40933 (local address 127.1.151.131:38675)
0808 06:05:01.265025 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:01.265066 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:01.266334 (+ 1268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:01.266862 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:01.266869 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:01.267206 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:01.267650 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:01.267663 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:01.268435 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:01.268438 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:01.269308 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:01.269317 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:01.269783 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:01.269787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:01.269946 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:01.270466 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:01.270481 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:01.272006 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:01.273657 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:01.273663 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:01.273676 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:01.273917 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:01.274200 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:01.274203 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:01.274206 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:01.274341 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:01.274722 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:01.274729 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:01.274950 (+ 221us) client_negotiation.cc:769] Sending connection context
0808 06:05:01.275105 (+ 155us) client_negotiation.cc:240] Negotiation successful
0808 06:05:01.275413 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":120,"threads_started":1}
I20260808 06:05:01.277154 11293 heartbeater.cc:347] Connected to a master server at 127.1.151.190:40933
I20260808 06:05:01.277367 11293 heartbeater.cc:464] Registering TS with master...
I20260808 06:05:01.277804 11293 heartbeater.cc:511] Master 127.1.151.190:40933 requested a full tablet report, sending...
I20260808 06:05:01.278759 10599 ts_manager.cc:194] Registered new tserver with Master: a71c9a2f7f8144c2ab708992233ac1dc (127.1.151.131:35965)
I20260808 06:05:01.279295 10599 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.131@KRBTEST.COM'} at 127.1.151.131:38675
I20260808 06:05:01.285030 1630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/krb5kdc/test-admin.keytab.
Aug 08 06:05:01 dist-test-slave-zj8q krb5kdc[10516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169101, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:01.302112 1630 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 06:05:01.304967 11300 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:01.303501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47640 (local address 127.1.151.129:36307)
0808 06:05:01.303853 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:01.303864 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:01.303887 (+ 23us) server_negotiation.cc:407] Connection header received
0808 06:05:01.303935 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:01.303941 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:01.304119 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:01.304273 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:01.304765 (+ 492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47640: BlockingRecv error: recv got EOF from 127.0.0.1:47640 (error 108)
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":125,"threads_started":1}
W20260808 06:05:01.305075 1630 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/krb5kdc/krb5cc' of type FILE
Aug 08 06:05:01 dist-test-slave-zj8q krb5kdc[10516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169101, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:01.306142 1630 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 06:05:01 dist-test-slave-zj8q krb5kdc[10516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169101, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:01.307045 1630 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 06:05:01 dist-test-slave-zj8q krb5kdc[10516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169101, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.129@KRBTEST.COM
I20260808 06:05:01.316231 11300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:01.307900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47644 (local address 127.1.151.129:36307)
0808 06:05:01.308015 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:01.308019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:01.308133 (+ 114us) server_negotiation.cc:407] Connection header received
0808 06:05:01.308202 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:01.308205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:01.308243 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:01.308339 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:01.308824 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:01.309595 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:01.310452 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:01.310806 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:01.310886 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:01.313142 (+ 2256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:01.313151 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:01.313184 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:01.313206 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:01.314708 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:01.315085 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:01.315089 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:01.315091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:01.315182 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:01.315370 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:01.315372 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:01.315373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:01.315613 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:01.315758 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:01.315909 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 06:05:01.316048 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260808 06:05:01.319692 11300 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:01.318808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47656 (local address 127.1.151.129:36307)
0808 06:05:01.318932 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:01.318936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:01.318994 (+ 58us) server_negotiation.cc:407] Connection header received
0808 06:05:01.319079 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:01.319081 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:01.319111 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:01.319171 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:01.319574 (+ 403us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47656: BlockingRecv error: recv got EOF from 127.0.0.1:47656 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260808 06:05:01.319787 1630 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/krb5kdc/krb5cc' of type FILE
Aug 08 06:05:01 dist-test-slave-zj8q krb5kdc[10516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169101, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:01.320612 1630 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 06:05:01 dist-test-slave-zj8q krb5kdc[10516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169101, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:01.321370 1630 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 06:05:01 dist-test-slave-zj8q krb5kdc[10516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169101, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.129@KRBTEST.COM
I20260808 06:05:01.328985 11300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:01.322107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47672 (local address 127.1.151.129:36307)
0808 06:05:01.322262 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:01.322265 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:01.322309 (+ 44us) server_negotiation.cc:407] Connection header received
0808 06:05:01.322390 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:01.322392 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:01.322421 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:01.322478 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:01.322901 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:01.323411 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:01.324148 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:01.324365 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:01.324431 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:01.326358 (+ 1927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:01.326361 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:01.326363 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:01.326384 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:01.327587 (+ 1203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:01.327914 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:01.327917 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:01.327918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:01.327982 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:01.328236 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:01.328239 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:01.328240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:01.328404 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:01.328509 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:01.328758 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 06:05:01.328845 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
W20260808 06:05:01.331301 11300 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:01.330583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47674 (local address 127.1.151.129:36307)
0808 06:05:01.330705 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:01.330709 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:01.330759 (+ 50us) server_negotiation.cc:407] Connection header received
0808 06:05:01.330844 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:01.330846 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:01.330876 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:01.330931 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:01.331214 (+ 283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47674: BlockingRecv error: recv got EOF from 127.0.0.1:47674 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 06:05:01.331384 1630 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786169070948234-1630-0/krb5kdc/krb5cc' of type FILE
Aug 08 06:05:01 dist-test-slave-zj8q krb5kdc[10516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169101, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:01.332316 1630 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 06:05:01 dist-test-slave-zj8q krb5kdc[10516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169101, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:01.333094 1630 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 06:05:01 dist-test-slave-zj8q krb5kdc[10516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169101, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.129@KRBTEST.COM
I20260808 06:05:01.340605 11300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:01.333818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47676 (local address 127.1.151.129:36307)
0808 06:05:01.333934 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:01.333937 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:01.333968 (+ 31us) server_negotiation.cc:407] Connection header received
0808 06:05:01.334062 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:01.334064 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:01.334094 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:01.334190 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:01.334680 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:01.335154 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:01.335905 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:01.336114 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:01.336185 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:01.338021 (+ 1836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:01.338024 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:01.338025 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:01.338046 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:01.339359 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:01.339776 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:01.339779 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:01.339780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:01.339852 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:01.340030 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:01.340032 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:01.340033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:01.340167 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:01.340249 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:01.340393 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 06:05:01.340473 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 06:05:01.341483 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 10696
I20260808 06:05:01.357324 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 10894
I20260808 06:05:01.364323 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 11101
I20260808 06:05:01.381477 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 10538
2026-08-08T06:05:01Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2264 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:05:01 dist-test-slave-zj8q krb5kdc[11323](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:05:01 dist-test-slave-zj8q krb5kdc[11323](info): set up 2 sockets
Aug 08 06:05:01 dist-test-slave-zj8q krb5kdc[11323](info): commencing operation
krb5kdc: starting...
W20260808 06:05:02.536369 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:05:02 dist-test-slave-zj8q krb5kdc[11323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:05:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:05:02Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.151.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.151.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.151.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.190.keytab.
Entry for principal oryx/127.1.151.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.190.keytab.
I20260808 06:05:02.654816 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:35293
--webserver_interface=127.1.151.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.190.keytab
--principal=oryx/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:45251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:35293
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:05:02.763373 11344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:02.763622 11344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:02.763686 11344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:02.767601 11344 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:05:02.767659 11344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:02.767686 11344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:02.767709 11344 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:05:02.767738 11344 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:05:02.772785 11344 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:45251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:35293
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.190.keytab
--principal=oryx/127.1.151.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:35293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:05:02.773754 11344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:02.774834 11344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:02.782842 11349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:02.782840 11352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:02.782850 11350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:02.783190 11344 server_base.cc:1034] running on GCE node
I20260808 06:05:02.783732 11344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:02.784790 11344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:02.785961 11344 hybrid_clock.cc:648] HybridClock initialized: now 1786169102785949 us; error 34 us; skew 500 ppm
Aug 08 06:05:02 dist-test-slave-zj8q krb5kdc[11323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169102, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:02.793155 11344 init.cc:377] Logged in from keytab as oryx/127.1.151.190@KRBTEST.COM (short username oryx)
I20260808 06:05:02.794021 11344 webserver.cc:460] Webserver started at http://127.1.151.190:42005/ using document root <none> and password file <none>
I20260808 06:05:02.794587 11344 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:02.794664 11344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:02.794868 11344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:05:02.796867 11344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "88cc958608974ee987b790f318d04d52"
format_stamp: "Formatted at 2026-08-08 06:05:02 on dist-test-slave-zj8q"
server_key: "f22ee289e4c5167c9718412c0cc5292c"
server_key_iv: "1679efd8ca1f49ef62689e8a37031ef5"
server_key_version: "encryptionkey@0"
I20260808 06:05:02.797406 11344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "88cc958608974ee987b790f318d04d52"
format_stamp: "Formatted at 2026-08-08 06:05:02 on dist-test-slave-zj8q"
server_key: "f22ee289e4c5167c9718412c0cc5292c"
server_key_iv: "1679efd8ca1f49ef62689e8a37031ef5"
server_key_version: "encryptionkey@0"
I20260808 06:05:02.801020 11344 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260808 06:05:02.803301 11360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:02.804415 11344 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 06:05:02.804525 11344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "88cc958608974ee987b790f318d04d52"
format_stamp: "Formatted at 2026-08-08 06:05:02 on dist-test-slave-zj8q"
server_key: "f22ee289e4c5167c9718412c0cc5292c"
server_key_iv: "1679efd8ca1f49ef62689e8a37031ef5"
server_key_version: "encryptionkey@0"
I20260808 06:05:02.804652 11344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:02.826108 11344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:02.828856 11344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:05:02.829000 11344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:02.835685 11344 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:35293
I20260808 06:05:02.835945 11443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:35293 every 8 connection(s)
I20260808 06:05:02.836573 11344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:05:02.839214 11444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:05:02.840734 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 11344
I20260808 06:05:02.840863 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:05:02.841130 1630 external_mini_cluster.cc:1428] Setting key d804c8a3ceef3c56bd326b0626ef0306
I20260808 06:05:02.845059 11444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52: Bootstrap starting.
I20260808 06:05:02.847404 11444 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52: Neither blocks nor log segments found. Creating new log.
I20260808 06:05:02.848201 11444 log.cc:824] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52: Log is configured to *not* fsync() on all Append() calls
Aug 08 06:05:02 dist-test-slave-zj8q krb5kdc[11323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM
I20260808 06:05:02.850208 11444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52: No bootstrap required, opened a new log
I20260808 06:05:02.853613 11444 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } }
I20260808 06:05:02.853830 11444 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:05:02.853868 11444 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88cc958608974ee987b790f318d04d52, State: Initialized, Role: FOLLOWER
I20260808 06:05:02.854187 11447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:02.842473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57770 (local address 127.1.151.190:35293)
0808 06:05:02.842890 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:02.842904 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:02.842937 (+ 33us) server_negotiation.cc:407] Connection header received
0808 06:05:02.843564 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:02.843580 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:02.843902 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:02.844189 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:02.844961 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:02.845976 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:02.846832 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:02.847307 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:02.847461 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:02.850028 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:02.850033 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:02.850046 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:02.850071 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:02.851605 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:02.851994 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:02.851999 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:02.852003 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:02.852116 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:02.852366 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:02.852370 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:02.852372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:02.852728 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:02.852922 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:02.853180 (+ 258us) server_negotiation.cc:299] Negotiation successful
0808 06:05:02.853436 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":130,"threads_started":1}
I20260808 06:05:02.854319 11444 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } }
I20260808 06:05:02.854470 11444 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:05:02.854532 11444 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:05:02.854625 11444 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:05:02.855530 11444 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } }
I20260808 06:05:02.855877 11444 leader_election.cc:302] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88cc958608974ee987b790f318d04d52; no voters:
I20260808 06:05:02.856181 11444 leader_election.cc:288] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:05:02.856269 11449 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:05:02.856475 11449 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [term 1 LEADER]: Becoming Leader. State: Replica: 88cc958608974ee987b790f318d04d52, State: Running, Role: LEADER
I20260808 06:05:02.856809 11449 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } }
I20260808 06:05:02.857252 11444 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:05:02.857827 11451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88cc958608974ee987b790f318d04d52. Latest consensus state: current_term: 1 leader_uuid: "88cc958608974ee987b790f318d04d52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } }
I20260808 06:05:02.858069 11451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [sys.catalog]: This master's current role is: LEADER
I20260808 06:05:02.858325 11450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88cc958608974ee987b790f318d04d52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } }
I20260808 06:05:02.858474 11450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [sys.catalog]: This master's current role is: LEADER
W20260808 06:05:02.861737 11466 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 06:05:02.861801 11466 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 06:05:02.861876 11460 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:05:02.862602 11460 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:05:02.867777 11460 catalog_manager.cc:1347] Generated new cluster ID: 34f12aa373e04701a013720e69244e25
I20260808 06:05:02.867838 11460 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:05:02.882313 11460 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:05:02.883406 11460 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:05:02.893348 11460 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52: Generated new TSK 0
I20260808 06:05:02.894045 11460 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.151.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.151.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.151.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.129.keytab.
Entry for principal oryx/127.1.151.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.129.keytab.
I20260808 06:05:02.923322 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.129:0
--local_ip_for_outbound_sockets=127.1.151.129
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:35293
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.129.keytab
--principal=oryx/127.1.151.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:45251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:05:03.034934 11475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:03.035178 11475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:03.035219 11475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:03.039477 11475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:03.039535 11475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:03.039600 11475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.129
I20260808 06:05:03.044902 11475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:45251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.129.keytab
--principal=oryx/127.1.151.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:35293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.129
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:05:03.045944 11475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:03.047138 11475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:03.056731 11483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:03.056674 11491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:03.056653 11482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:03.056926 11475 server_base.cc:1034] running on GCE node
I20260808 06:05:03.057437 11475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:03.058082 11475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:03.059268 11475 hybrid_clock.cc:648] HybridClock initialized: now 1786169103059234 us; error 39 us; skew 500 ppm
Aug 08 06:05:03 dist-test-slave-zj8q krb5kdc[11323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169103, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:03.061966 11475 init.cc:377] Logged in from keytab as oryx/127.1.151.129@KRBTEST.COM (short username oryx)
I20260808 06:05:03.062840 11475 webserver.cc:460] Webserver started at http://127.1.151.129:41815/ using document root <none> and password file <none>
I20260808 06:05:03.063373 11475 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:03.063439 11475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:03.063625 11475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:05:03.065646 11475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-0/data/instance:
uuid: "a67dd2b3f83143dd958e5cb489a649c2"
format_stamp: "Formatted at 2026-08-08 06:05:03 on dist-test-slave-zj8q"
server_key: "e175e1031493194e83dbb9689a81e85b"
server_key_iv: "0aa7960da8ffdc86bcc1e20442539200"
server_key_version: "encryptionkey@0"
I20260808 06:05:03.066218 11475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance:
uuid: "a67dd2b3f83143dd958e5cb489a649c2"
format_stamp: "Formatted at 2026-08-08 06:05:03 on dist-test-slave-zj8q"
server_key: "e175e1031493194e83dbb9689a81e85b"
server_key_iv: "0aa7960da8ffdc86bcc1e20442539200"
server_key_version: "encryptionkey@0"
I20260808 06:05:03.069775 11475 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 06:05:03.072201 11501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:03.073374 11475 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 06:05:03.073494 11475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-0/wal
uuid: "a67dd2b3f83143dd958e5cb489a649c2"
format_stamp: "Formatted at 2026-08-08 06:05:03 on dist-test-slave-zj8q"
server_key: "e175e1031493194e83dbb9689a81e85b"
server_key_iv: "0aa7960da8ffdc86bcc1e20442539200"
server_key_version: "encryptionkey@0"
I20260808 06:05:03.073582 11475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:03.089258 11475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:03.092237 11475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:05:03.092409 11475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:03.092974 11475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:05:03.093788 11475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:05:03.093837 11475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:03.093887 11475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:05:03.093912 11475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:03.102582 11475 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.129:38287
I20260808 06:05:03.102636 11681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.129:38287 every 8 connection(s)
I20260808 06:05:03.103454 11475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
I20260808 06:05:03.109688 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 11475
I20260808 06:05:03.109804 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance
I20260808 06:05:03.110114 1630 external_mini_cluster.cc:1428] Setting key cb5fcb293eb93364a9f19342b0abc271
Aug 08 06:05:03 dist-test-slave-zj8q krb5kdc[11323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169103, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM
I20260808 06:05:03.117513 11447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:03.105170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:56715 (local address 127.1.151.190:35293)
0808 06:05:03.105286 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:03.105290 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:03.106114 (+ 824us) server_negotiation.cc:407] Connection header received
0808 06:05:03.107438 (+ 1324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:03.107442 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:03.107492 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:03.107565 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:03.108774 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.109312 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.110607 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.110845 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.110937 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:03.113603 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:03.113607 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:03.113610 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:03.113661 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:03.115254 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:03.115895 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:03.115899 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:03.115900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:03.115983 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:03.116359 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:03.116362 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:03.116363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:03.116542 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:03.116625 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:03.117266 (+ 641us) server_negotiation.cc:299] Negotiation successful
0808 06:05:03.117384 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 06:05:03.118626 11686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:03.105483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35293 (local address 127.1.151.129:56715)
0808 06:05:03.105963 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:03.106003 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:03.107215 (+ 1212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:03.107777 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:03.107783 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:03.108126 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:03.108624 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:03.108640 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.109436 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.109456 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:03.110493 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:03.110503 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.110984 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.110988 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:03.111148 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:03.111712 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:03.111728 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:03.113394 (+ 1666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:03.115377 (+ 1983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:03.115384 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:03.115398 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:03.115759 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:03.116077 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:03.116080 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:03.116082 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:03.116264 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:03.116661 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:03.116669 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:03.116983 (+ 314us) client_negotiation.cc:769] Sending connection context
0808 06:05:03.117209 (+ 226us) client_negotiation.cc:240] Negotiation successful
0808 06:05:03.117549 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":135,"threads_started":1}
I20260808 06:05:03.119903 11682 heartbeater.cc:347] Connected to a master server at 127.1.151.190:35293
I20260808 06:05:03.120157 11682 heartbeater.cc:464] Registering TS with master...
I20260808 06:05:03.120627 11682 heartbeater.cc:511] Master 127.1.151.190:35293 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.1.151.130@KRBTEST.COM; defaulting to no policy
I20260808 06:05:03.122210 11384 ts_manager.cc:194] Registered new tserver with Master: a67dd2b3f83143dd958e5cb489a649c2 (127.1.151.129:38287)
I20260808 06:05:03.123343 11384 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.151.129@KRBTEST.COM'} at 127.1.151.129:56715
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.151.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.151.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.130.keytab.
Entry for principal oryx/127.1.151.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.130.keytab.
I20260808 06:05:03.138808 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.130:0
--local_ip_for_outbound_sockets=127.1.151.130
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:35293
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.130.keytab
--principal=oryx/127.1.151.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:45251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:05:03.249325 11690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:03.249568 11690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:03.249629 11690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:03.254315 11690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:03.254375 11690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:03.254462 11690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.130
I20260808 06:05:03.260346 11690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:45251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.130.keytab
--principal=oryx/127.1.151.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:35293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.130
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:05:03.261447 11690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:03.262498 11690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:03.271663 11695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:03.271684 11698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:03.271699 11690 server_base.cc:1034] running on GCE node
W20260808 06:05:03.271663 11700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:03.272239 11690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:03.272871 11690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:03.274031 11690 hybrid_clock.cc:648] HybridClock initialized: now 1786169103274005 us; error 36 us; skew 500 ppm
Aug 08 06:05:03 dist-test-slave-zj8q krb5kdc[11323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169103, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:03.276767 11690 init.cc:377] Logged in from keytab as oryx/127.1.151.130@KRBTEST.COM (short username oryx)
I20260808 06:05:03.277613 11690 webserver.cc:460] Webserver started at http://127.1.151.130:46883/ using document root <none> and password file <none>
I20260808 06:05:03.278126 11690 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:03.278224 11690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:03.278429 11690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:05:03.280441 11690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-1/data/instance:
uuid: "1f756da4d21243bfbc204b1f2950800b"
format_stamp: "Formatted at 2026-08-08 06:05:03 on dist-test-slave-zj8q"
server_key: "7413aca3209fb65c0355047c7119ecbb"
server_key_iv: "67d9aad79823dbceaa403091193b4fed"
server_key_version: "encryptionkey@0"
I20260808 06:05:03.281106 11690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance:
uuid: "1f756da4d21243bfbc204b1f2950800b"
format_stamp: "Formatted at 2026-08-08 06:05:03 on dist-test-slave-zj8q"
server_key: "7413aca3209fb65c0355047c7119ecbb"
server_key_iv: "67d9aad79823dbceaa403091193b4fed"
server_key_version: "encryptionkey@0"
I20260808 06:05:03.284652 11690 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 06:05:03.287266 11711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:03.288434 11690 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 06:05:03.288544 11690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-1/wal
uuid: "1f756da4d21243bfbc204b1f2950800b"
format_stamp: "Formatted at 2026-08-08 06:05:03 on dist-test-slave-zj8q"
server_key: "7413aca3209fb65c0355047c7119ecbb"
server_key_iv: "67d9aad79823dbceaa403091193b4fed"
server_key_version: "encryptionkey@0"
I20260808 06:05:03.288678 11690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:03.307869 11690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:03.310909 11690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:05:03.311081 11690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:03.311652 11690 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:05:03.312430 11690 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:05:03.312482 11690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:03.312533 11690 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:05:03.312562 11690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:03.320852 11690 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.130:38501
I20260808 06:05:03.320888 11888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.130:38501 every 8 connection(s)
I20260808 06:05:03.321676 11690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
I20260808 06:05:03.324860 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 11690
I20260808 06:05:03.324995 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance
I20260808 06:05:03.325311 1630 external_mini_cluster.cc:1428] Setting key 5e3986890ab59c76297f2e565b33c691
Aug 08 06:05:03 dist-test-slave-zj8q krb5kdc[11323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169103, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM
I20260808 06:05:03.336014 11447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:03.323374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:57603 (local address 127.1.151.190:35293)
0808 06:05:03.323529 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:03.323534 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:03.324355 (+ 821us) server_negotiation.cc:407] Connection header received
0808 06:05:03.325959 (+ 1604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:03.325963 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:03.326016 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:03.326094 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:03.327568 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.328102 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.329494 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.329820 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.329930 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:03.332631 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:03.332634 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:03.332637 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:03.332664 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:03.333960 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:03.334551 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:03.334553 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:03.334554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:03.334627 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:03.334978 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:03.334980 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:03.334981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:03.335146 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:03.335216 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:03.335797 (+ 581us) server_negotiation.cc:299] Negotiation successful
0808 06:05:03.335894 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
WARNING: no policy specified for oryx/127.1.151.131@KRBTEST.COM; defaulting to no policy
I20260808 06:05:03.336946 11893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:03.323741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35293 (local address 127.1.151.130:57603)
0808 06:05:03.324207 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:03.324254 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:03.325764 (+ 1510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:03.326319 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:03.326328 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:03.326761 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:03.327398 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:03.327429 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.328229 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.328232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:03.329348 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:03.329364 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.329988 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.329991 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:03.330144 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:03.330698 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:03.330714 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:03.332460 (+ 1746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:03.334074 (+ 1614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:03.334080 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:03.334094 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:03.334461 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:03.334717 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:03.334720 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:03.334723 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:03.334897 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:03.335245 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:03.335251 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:03.335554 (+ 303us) client_negotiation.cc:769] Sending connection context
0808 06:05:03.335769 (+ 215us) client_negotiation.cc:240] Negotiation successful
0808 06:05:03.336079 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":115,"threads_started":1}
I20260808 06:05:03.338197 11889 heartbeater.cc:347] Connected to a master server at 127.1.151.190:35293
I20260808 06:05:03.338428 11889 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.151.131@KRBTEST.COM" created.
I20260808 06:05:03.338927 11889 heartbeater.cc:511] Master 127.1.151.190:35293 requested a full tablet report, sending...
I20260808 06:05:03.340022 11384 ts_manager.cc:194] Registered new tserver with Master: 1f756da4d21243bfbc204b1f2950800b (127.1.151.130:38501)
I20260808 06:05:03.340606 11384 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.151.130@KRBTEST.COM'} at 127.1.151.130:57603
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.151.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.131.keytab.
Entry for principal oryx/127.1.151.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.131.keytab.
I20260808 06:05:03.353017 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.131:0
--local_ip_for_outbound_sockets=127.1.151.131
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:35293
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.131.keytab
--principal=oryx/127.1.151.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:45251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:05:03.462388 11900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:03.462623 11900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:03.462663 11900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:03.466588 11900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:03.466635 11900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:03.466696 11900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.131
I20260808 06:05:03.471784 11900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:45251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.131.keytab
--principal=oryx/127.1.151.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:35293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.131
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:05:03.472708 11900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:03.473621 11900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:03.482951 11909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:03.482950 11916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:03.483096 11900 server_base.cc:1034] running on GCE node
W20260808 06:05:03.483259 11910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:03.483631 11900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:03.484246 11900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:03.485405 11900 hybrid_clock.cc:648] HybridClock initialized: now 1786169103485393 us; error 30 us; skew 500 ppm
Aug 08 06:05:03 dist-test-slave-zj8q krb5kdc[11323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169103, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:03.487953 11900 init.cc:377] Logged in from keytab as oryx/127.1.151.131@KRBTEST.COM (short username oryx)
I20260808 06:05:03.488756 11900 webserver.cc:460] Webserver started at http://127.1.151.131:45859/ using document root <none> and password file <none>
I20260808 06:05:03.489284 11900 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:03.489351 11900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:03.489549 11900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:05:03.491526 11900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-2/data/instance:
uuid: "543ae60e604c41868b2b7ab4a435673b"
format_stamp: "Formatted at 2026-08-08 06:05:03 on dist-test-slave-zj8q"
server_key: "3ed464d246348e3a0a1a531c16c4b84a"
server_key_iv: "c9aabb5229b211c594326ef2d5a6e467"
server_key_version: "encryptionkey@0"
I20260808 06:05:03.492063 11900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance:
uuid: "543ae60e604c41868b2b7ab4a435673b"
format_stamp: "Formatted at 2026-08-08 06:05:03 on dist-test-slave-zj8q"
server_key: "3ed464d246348e3a0a1a531c16c4b84a"
server_key_iv: "c9aabb5229b211c594326ef2d5a6e467"
server_key_version: "encryptionkey@0"
I20260808 06:05:03.495666 11900 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 06:05:03.497936 11928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:03.499176 11900 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:05:03.499286 11900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-2/wal
uuid: "543ae60e604c41868b2b7ab4a435673b"
format_stamp: "Formatted at 2026-08-08 06:05:03 on dist-test-slave-zj8q"
server_key: "3ed464d246348e3a0a1a531c16c4b84a"
server_key_iv: "c9aabb5229b211c594326ef2d5a6e467"
server_key_version: "encryptionkey@0"
I20260808 06:05:03.499374 11900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:03.509552 11900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:03.512188 11900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:05:03.512352 11900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:03.512897 11900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:05:03.513700 11900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:05:03.513747 11900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:03.513798 11900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:05:03.513824 11900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:03.522166 11900 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.131:33501
I20260808 06:05:03.522192 12099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.131:33501 every 8 connection(s)
I20260808 06:05:03.522996 11900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
I20260808 06:05:03.529289 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 11900
I20260808 06:05:03.529385 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance
I20260808 06:05:03.529647 1630 external_mini_cluster.cc:1428] Setting key 14fe4ef86c1ea410203079363cee9260
Aug 08 06:05:03 dist-test-slave-zj8q krb5kdc[11323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169103, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM
I20260808 06:05:03.535766 11447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:03.524662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:34983 (local address 127.1.151.190:35293)
0808 06:05:03.524821 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:03.524824 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:03.525657 (+ 833us) server_negotiation.cc:407] Connection header received
0808 06:05:03.526921 (+ 1264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:03.526924 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:03.526972 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:03.527044 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:03.528152 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.528682 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.529852 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.530097 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.530240 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:03.532740 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:03.532743 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:03.532745 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:03.532769 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:03.534027 (+ 1258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:03.534482 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:03.534485 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:03.534486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:03.534558 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:03.534896 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:03.534897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:03.534898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:03.535053 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:03.535121 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:03.535554 (+ 433us) server_negotiation.cc:299] Negotiation successful
0808 06:05:03.535645 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 06:05:03.536489 12104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:03.525034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35293 (local address 127.1.151.131:34983)
0808 06:05:03.525482 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:03.525536 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:03.526725 (+ 1189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:03.527187 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:03.527193 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:03.527538 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:03.528018 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:03.528032 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.528772 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.528774 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:03.529735 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:03.529745 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.530287 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.530291 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:03.530488 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:03.531050 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:03.531070 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:03.532579 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:03.534111 (+ 1532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:03.534115 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:03.534125 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:03.534409 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:03.534639 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:03.534642 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:03.534644 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:03.534817 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:03.535160 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:03.535165 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:03.535374 (+ 209us) client_negotiation.cc:769] Sending connection context
0808 06:05:03.535521 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 06:05:03.535752 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":129,"threads_started":1}
I20260808 06:05:03.537475 12101 heartbeater.cc:347] Connected to a master server at 127.1.151.190:35293
I20260808 06:05:03.537686 12101 heartbeater.cc:464] Registering TS with master...
I20260808 06:05:03.538128 12101 heartbeater.cc:511] Master 127.1.151.190:35293 requested a full tablet report, sending...
I20260808 06:05:03.539137 11384 ts_manager.cc:194] Registered new tserver with Master: 543ae60e604c41868b2b7ab4a435673b (127.1.151.131:33501)
I20260808 06:05:03.539672 11384 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.151.131@KRBTEST.COM'} at 127.1.151.131:34983
I20260808 06:05:03.543278 1630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.151.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.151.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.151.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.189.keytab.
Entry for principal oryx/127.1.151.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.189.keytab.
I20260808 06:05:03.569247 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.189:44883
--webserver_interface=127.1.151.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.189.keytab
--principal=oryx/127.1.151.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:45251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:35293,127.1.151.189:44883
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:05:03.678241 12109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:03.678469 12109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:03.678514 12109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:03.682384 12109 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:05:03.682431 12109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:03.682449 12109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:03.682466 12109 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:05:03.682492 12109 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:05:03.687541 12109 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:45251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:35293,127.1.151.189:44883
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.189.keytab
--principal=oryx/127.1.151.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.189:44883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.151.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
W20260808 06:05:03.687726 12109 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.1.151.190:35293,127.1.151.189:44883'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 06:05:03.688491 12109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:03.689450 12109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:03.697419 12117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:03.697432 12123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:03.697430 12118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:03.697616 12109 server_base.cc:1034] running on GCE node
I20260808 06:05:03.698297 12109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:03.699383 12109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:03.700541 12109 hybrid_clock.cc:648] HybridClock initialized: now 1786169103700524 us; error 30 us; skew 500 ppm
Aug 08 06:05:03 dist-test-slave-zj8q krb5kdc[11323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169103, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:03.703249 12109 init.cc:377] Logged in from keytab as oryx/127.1.151.189@KRBTEST.COM (short username oryx)
I20260808 06:05:03.704095 12109 webserver.cc:460] Webserver started at http://127.1.151.189:44359/ using document root <none> and password file <none>
I20260808 06:05:03.704617 12109 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:03.704686 12109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:03.704895 12109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:05:03.706962 12109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-1/data/instance:
uuid: "81fbba8a10f54078bade0f78cc102007"
format_stamp: "Formatted at 2026-08-08 06:05:03 on dist-test-slave-zj8q"
server_key: "3b8e708897585a0ed32cc8cf8842eae0"
server_key_iv: "c1537ecf126d40547e60c335c945b340"
server_key_version: "encryptionkey@0"
I20260808 06:05:03.707504 12109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-1/wal/instance:
uuid: "81fbba8a10f54078bade0f78cc102007"
format_stamp: "Formatted at 2026-08-08 06:05:03 on dist-test-slave-zj8q"
server_key: "3b8e708897585a0ed32cc8cf8842eae0"
server_key_iv: "c1537ecf126d40547e60c335c945b340"
server_key_version: "encryptionkey@0"
I20260808 06:05:03.711067 12109 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 06:05:03.713403 12133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:03.714555 12109 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:05:03.714669 12109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-1/wal
uuid: "81fbba8a10f54078bade0f78cc102007"
format_stamp: "Formatted at 2026-08-08 06:05:03 on dist-test-slave-zj8q"
server_key: "3b8e708897585a0ed32cc8cf8842eae0"
server_key_iv: "c1537ecf126d40547e60c335c945b340"
server_key_version: "encryptionkey@0"
I20260808 06:05:03.714756 12109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:03.724258 12109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:03.726830 12109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:05:03.726995 12109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:03.733337 12109 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.189:44883
I20260808 06:05:03.733600 12219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.189:44883 every 8 connection(s)
I20260808 06:05:03.734248 12109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-1/data/info.pb
I20260808 06:05:03.735376 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 12109
I20260808 06:05:03.737342 12223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:05:03.741720 12223 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 06:05:03 dist-test-slave-zj8q krb5kdc[11323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.151.189@KRBTEST.COM
Aug 08 06:05:03 dist-test-slave-zj8q krb5kdc[11323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169103, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.189@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM
I20260808 06:05:03.750628 12228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:03.736851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59540 (local address 127.1.151.189:44883)
0808 06:05:03.737244 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:03.737262 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:03.737301 (+ 39us) server_negotiation.cc:407] Connection header received
0808 06:05:03.738126 (+ 825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:03.738148 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:03.738554 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:03.738908 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:03.739565 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.741049 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.741900 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.742570 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.742746 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:03.746054 (+ 3308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:03.746060 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:03.746072 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:03.746102 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:03.747798 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:03.748198 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:03.748204 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:03.748210 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:03.748347 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:03.748643 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:03.748647 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:03.748648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:03.749045 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:03.749256 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:03.749529 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 06:05:03.749795 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":115,"threads_started":1}
I20260808 06:05:03.752007 11447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:03.743627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57772 (local address 127.1.151.190:35293)
0808 06:05:03.743751 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:03.743754 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:03.744076 (+ 322us) server_negotiation.cc:407] Connection header received
0808 06:05:03.744207 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:03.744210 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:03.744254 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:03.744314 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:03.744959 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.745455 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.746490 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.746716 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.746804 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:03.748839 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:03.748842 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:03.748844 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:03.748868 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:03.750266 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:03.750762 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:03.750766 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:03.750768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:03.750854 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:03.751143 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:03.751146 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:03.751147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:03.751355 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:03.751439 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:03.751758 (+ 319us) server_negotiation.cc:299] Negotiation successful
0808 06:05:03.751858 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 06:05:03.752032 12229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:03.743595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35293 (local address 127.0.0.1:57772)
0808 06:05:03.743959 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:03.743996 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:03.744110 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:03.744377 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:03.744382 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:03.744698 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:03.744817 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:03.744823 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.745572 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.745575 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:03.746367 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:03.746378 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.746820 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.746824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:03.746934 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:03.747307 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:03.747318 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:03.748713 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:03.750362 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:03.750370 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:03.750375 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:03.750654 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:03.750958 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:03.750961 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:03.750963 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:03.751047 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:03.751460 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:03.751466 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:03.751567 (+ 101us) client_negotiation.cc:769] Sending connection context
0808 06:05:03.751686 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 06:05:03.751859 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":150,"threads_started":1}
I20260808 06:05:03.753142 12223 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 06:05:03 dist-test-slave-zj8q krb5kdc[11323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169103, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.189@KRBTEST.COM for oryx/127.1.151.189@KRBTEST.COM
I20260808 06:05:03.760047 12229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:03.753547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.189:44883 (local address 127.0.0.1:59550)
0808 06:05:03.753644 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:03.753655 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:03.753713 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:03.753888 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:03.753890 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:03.754006 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:03.754089 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:03.754093 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.755134 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.755137 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:03.755625 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:03.755632 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.755986 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.755989 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:03.756054 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:03.756383 (+ 329us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:03.756392 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:03.757422 (+ 1030us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:03.758814 (+ 1392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:03.758817 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:03.758819 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:03.759004 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:03.759230 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:03.759233 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:03.759234 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:03.759305 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:03.759606 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:03.759609 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:03.759688 (+ 79us) client_negotiation.cc:769] Sending connection context
0808 06:05:03.759769 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 06:05:03.759889 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 06:05:03.760067 12228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:03.753638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59550 (local address 127.1.151.189:44883)
0808 06:05:03.753741 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:03.753745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:03.753755 (+ 10us) server_negotiation.cc:407] Connection header received
0808 06:05:03.753782 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:03.753784 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:03.753813 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:03.753872 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:03.754194 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.755016 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.755708 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.755914 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.755982 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:03.757571 (+ 1589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:03.757574 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:03.757576 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:03.757595 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:03.758726 (+ 1131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:03.759083 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:03.759086 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:03.759088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:03.759155 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:03.759377 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:03.759379 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:03.759381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:03.759525 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:03.759610 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:03.759822 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 06:05:03.759943 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":38}
I20260808 06:05:03.762574 12223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007: Bootstrap starting.
I20260808 06:05:03.764726 12223 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007: Neither blocks nor log segments found. Creating new log.
I20260808 06:05:03.765395 12223 log.cc:824] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007: Log is configured to *not* fsync() on all Append() calls
I20260808 06:05:03.767144 12223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007: No bootstrap required, opened a new log
I20260808 06:05:03.769944 12223 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } }
I20260808 06:05:03.770126 12223 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:05:03.770205 12223 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81fbba8a10f54078bade0f78cc102007, State: Initialized, Role: FOLLOWER
I20260808 06:05:03.770593 12223 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } }
I20260808 06:05:03.771013 12233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } } }
I20260808 06:05:03.771136 12233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 06:05:03.771670 12223 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 06:05:03.775909 12252 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 06:05:03.776013 12252 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 06:05:03.778348 12109 master_runner.cc:428] Detected that this master 81fbba8a10f54078bade0f78cc102007 is joining an existing cluster
I20260808 06:05:03.778406 12109 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.151.189:44883
I20260808 06:05:03.788465 11447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:03.780296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57788 (local address 127.1.151.190:35293)
0808 06:05:03.780412 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:03.780416 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:03.780720 (+ 304us) server_negotiation.cc:407] Connection header received
0808 06:05:03.781092 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:03.781095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:03.781155 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:03.781207 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:03.781771 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.782316 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.783609 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.783949 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.784064 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:03.785159 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:03.785163 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:03.785165 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:03.785191 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:03.786753 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:03.787279 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:03.787282 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:03.787284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:03.787383 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:03.787652 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:03.787655 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:03.787657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:03.787860 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:03.787948 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:03.788226 (+ 278us) server_negotiation.cc:299] Negotiation successful
0808 06:05:03.788332 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 06:05:03.788470 12262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:03.780248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35293 (local address 127.0.0.1:57788)
0808 06:05:03.780638 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:03.780652 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:03.780791 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:03.781345 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:03.781348 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:03.781536 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:03.781666 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:03.781671 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.782421 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.782425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:03.783479 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:03.783490 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.784066 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.784069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:03.784142 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:03.784582 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:03.784597 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:03.785002 (+ 405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:03.786871 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:03.786874 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:03.786876 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:03.787171 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:03.787482 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:03.787485 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:03.787487 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:03.787574 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:03.787945 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:03.787947 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:03.788040 (+ 93us) client_negotiation.cc:769] Sending connection context
0808 06:05:03.788151 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 06:05:03.788281 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":83,"spinlock_wait_cycles":1920,"thread_start_us":129,"threads_started":1}
I20260808 06:05:03.789041 12263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:03.780501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.189:44883 (local address 127.0.0.1:59554)
0808 06:05:03.780754 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:03.780765 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:03.780843 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:03.781073 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:03.781076 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:03.781288 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:03.781432 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:03.781439 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.782451 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.782454 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:03.783479 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:03.783490 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.784605 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.784609 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:03.784732 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:03.785420 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:03.785430 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:03.785840 (+ 410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:03.787607 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:03.787609 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:03.787611 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:03.787798 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:03.788051 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:03.788054 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:03.788056 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:03.788149 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:03.788532 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:03.788534 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:03.788613 (+ 79us) client_negotiation.cc:769] Sending connection context
0808 06:05:03.788722 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 06:05:03.788839 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":68,"thread_start_us":101,"threads_started":1}
I20260808 06:05:03.789055 12228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:03.780311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59554 (local address 127.1.151.189:44883)
0808 06:05:03.780410 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:03.780415 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:03.780837 (+ 422us) server_negotiation.cc:407] Connection header received
0808 06:05:03.780919 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:03.780922 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:03.780965 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:03.781045 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:03.781537 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.782357 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.784189 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.784503 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.784616 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:03.785979 (+ 1363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:03.785982 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:03.785984 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:03.786007 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:03.787505 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:03.787877 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:03.787880 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:03.787882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:03.787959 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:03.788245 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:03.788248 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:03.788250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:03.788421 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:03.788481 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:03.788793 (+ 312us) server_negotiation.cc:299] Negotiation successful
0808 06:05:03.788899 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":38}
Aug 08 06:05:03 dist-test-slave-zj8q krb5kdc[11323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169102, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.190@KRBTEST.COM for oryx/127.1.151.189@KRBTEST.COM
I20260808 06:05:03.801682 12266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:03.792952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.189:44883 (local address 127.0.0.1:59556)
0808 06:05:03.793622 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:03.793672 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:03.793777 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:03.794029 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:03.794033 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:03.794386 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:03.794492 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:03.794498 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.795401 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.795403 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:03.796019 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:03.796028 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.796549 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.796552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:03.796662 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:03.797101 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:03.797111 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:03.798670 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:03.800285 (+ 1615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:03.800294 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:03.800299 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:03.800513 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:03.800756 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:03.800758 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:03.800759 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:03.800832 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:03.801208 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:03.801212 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:03.801308 (+ 96us) client_negotiation.cc:769] Sending connection context
0808 06:05:03.801416 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 06:05:03.801556 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":96,"threads_started":1}
I20260808 06:05:03.801733 12228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:03.793023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59556 (local address 127.1.151.189:44883)
0808 06:05:03.793115 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:03.793119 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:03.793738 (+ 619us) server_negotiation.cc:407] Connection header received
0808 06:05:03.793882 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:03.793884 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:03.793931 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:03.794008 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:03.794587 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.795310 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.796116 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.796437 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.796522 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:03.798800 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:03.798804 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:03.798806 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:03.798830 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:03.800185 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:03.800592 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:03.800595 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:03.800597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:03.800682 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:03.800902 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:03.800905 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:03.800907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:03.801123 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:03.801212 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:03.801476 (+ 264us) server_negotiation.cc:299] Negotiation successful
0808 06:05:03.801579 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 06:05:03.803131 11384 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.151.189:44883: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "81fbba8a10f54078bade0f78cc102007"
member_type: NON_VOTER
last_known_addr {
host: "127.1.151.189"
port: 44883
}
attrs {
promote: true
}
}
dest_uuid: "88cc958608974ee987b790f318d04d52"
cas_config_opid_index: -1
I20260808 06:05:03.803805 11384 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: NON_VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: true } }
I20260808 06:05:03.805305 12270 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 81fbba8a10f54078bade0f78cc102007 (127.1.151.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: NON_VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: true } } }
I20260808 06:05:03.806105 12181 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:05:03.807267 12270 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.151.189:44883
I20260808 06:05:03.807438 12181 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [term 1 FOLLOWER]: Refusing update from remote peer 88cc958608974ee987b790f318d04d52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 06:05:03.807544 12270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "88cc958608974ee987b790f318d04d52" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: NON_VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: true } } }
I20260808 06:05:03.807624 12270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [sys.catalog]: This master's current role is: LEADER
I20260808 06:05:03.807742 12109 master.cc:537] Master@127.1.151.189:44883 shutting down...
I20260808 06:05:03.808197 12269 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: NON_VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 06:05:03.810015 11364 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260808 06:05:03.811165 11364 consensus_peers.cc:487] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 -> Peer 81fbba8a10f54078bade0f78cc102007 (127.1.151.189:44883): Couldn't send request to peer 81fbba8a10f54078bade0f78cc102007. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 06:05:03.811815 12109 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:05:03.812065 12109 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:05:03.812166 12109 tablet_replica.cc:330] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007: stopping tablet replica
WARNING: no policy specified for oryx/127.1.151.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.151.188@KRBTEST.COM" created.
I20260808 06:05:03.827334 12109 master.cc:552] Master@127.1.151.189:44883 shutdown complete.
I20260808 06:05:03.827392 12109 master_runner.cc:305] Clearing existing system tablet
I20260808 06:05:03.827646 12109 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:03.827705 12109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:03.829034 12109 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 06:05:03.830106 12278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:03.830317 12109 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:03.830441 12109 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 06:05:03.830497 12109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-1/wal
uuid: "81fbba8a10f54078bade0f78cc102007"
format_stamp: "Formatted at 2026-08-08 06:05:03 on dist-test-slave-zj8q"
server_key: "3b8e708897585a0ed32cc8cf8842eae0"
server_key_iv: "c1537ecf126d40547e60c335c945b340"
server_key_version: "encryptionkey@0"
I20260808 06:05:03.832553 12109 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007: Deleting tablet data with delete state TABLET_DATA_DELETED
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.151.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.188.keytab.
Entry for principal oryx/127.1.151.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.188.keytab.
I20260808 06:05:03.834892 12109 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 06:05:03.834977 12109 log.cc:1196] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 06:05:03.835232 12109 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007: Deleting consensus metadata
I20260808 06:05:03.835386 12109 master_runner.cc:315] Copying system tablet from 127.1.151.190:35293
I20260808 06:05:03.835448 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.188:33465
--webserver_interface=127.1.151.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.188.keytab
--principal=oryx/127.1.151.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:45251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:35293,127.1.151.189:44883,127.1.151.188:33465
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 06:05:03.836479 12109 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007: tablet copy: Beginning tablet copy session from remote peer at address 127.1.151.190:35293
I20260808 06:05:03.845293 11447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:03.837093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57804 (local address 127.1.151.190:35293)
0808 06:05:03.837246 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:03.837251 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:03.837613 (+ 362us) server_negotiation.cc:407] Connection header received
0808 06:05:03.837731 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:03.837734 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:03.837788 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:03.837873 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:03.838371 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.839195 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.840317 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.840663 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.840753 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:03.841929 (+ 1176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:03.841933 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:03.841936 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:03.841962 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:03.843564 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:03.844062 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:03.844065 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:03.844067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:03.844147 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:03.844446 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:03.844449 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:03.844451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:03.844676 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:03.844771 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:03.845030 (+ 259us) server_negotiation.cc:299] Negotiation successful
0808 06:05:03.845141 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 06:05:03.845314 12287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:03.837222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35293 (local address 127.0.0.1:57804)
0808 06:05:03.837507 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:03.837521 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:03.837647 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:03.837898 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:03.837901 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:03.838100 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:03.838258 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:03.838265 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.839302 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.839306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:03.840206 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:03.840218 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:03.840766 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:03.840770 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:03.840865 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:03.841376 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:03.841391 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:03.841796 (+ 405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:03.843698 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:03.843701 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:03.843703 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:03.843937 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:03.844246 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:03.844250 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:03.844252 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:03.844349 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:03.844766 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:03.844769 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:03.844867 (+ 98us) client_negotiation.cc:769] Sending connection context
0808 06:05:03.844971 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 06:05:03.845110 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":113,"threads_started":1}
I20260808 06:05:03.846223 11413 tablet_copy_service.cc:140] P 88cc958608974ee987b790f318d04d52: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 81fbba8a10f54078bade0f78cc102007 ({username='oryx', principal='oryx/127.1.151.189@KRBTEST.COM'} at 127.0.0.1:57804)
I20260808 06:05:03.846390 11413 tablet_copy_service.cc:161] P 88cc958608974ee987b790f318d04d52: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 81fbba8a10f54078bade0f78cc102007 at {username='oryx', principal='oryx/127.1.151.189@KRBTEST.COM'} at 127.0.0.1:57804: session id = 81fbba8a10f54078bade0f78cc102007-00000000000000000000000000000000
I20260808 06:05:03.847998 11413 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 06:05:03.849586 12109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 06:05:03.851830 12109 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007: tablet copy: Starting download of 0 data blocks...
I20260808 06:05:03.852011 12109 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007: tablet copy: Starting download of 1 WAL segments...
I20260808 06:05:03.853708 12109 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 06:05:03.855726 11413 tablet_copy_service.cc:342] P 88cc958608974ee987b790f318d04d52: Request end of tablet copy session 81fbba8a10f54078bade0f78cc102007-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.151.189@KRBTEST.COM'} at 127.0.0.1:57804
I20260808 06:05:03.855840 11413 tablet_copy_service.cc:433] P 88cc958608974ee987b790f318d04d52: ending tablet copy session 81fbba8a10f54078bade0f78cc102007-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 81fbba8a10f54078bade0f78cc102007
I20260808 06:05:03.863910 12109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:03.865442 12294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:03.865654 12290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:03.865715 12291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:03.865904 12109 server_base.cc:1034] running on GCE node
I20260808 06:05:03.866204 12109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:03.866484 12109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:03.867631 12109 hybrid_clock.cc:648] HybridClock initialized: now 1786169103867613 us; error 35 us; skew 500 ppm
Aug 08 06:05:03 dist-test-slave-zj8q krb5kdc[11323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169103, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:03.869132 12109 init.cc:377] Logged in from keytab as oryx/127.1.151.189@KRBTEST.COM (short username oryx)
I20260808 06:05:03.869552 12109 webserver.cc:460] Webserver started at http://127.1.151.189:40325/ using document root <none> and password file <none>
I20260808 06:05:03.869735 12109 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:03.869774 12109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:03.871040 12109 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 06:05:03.871991 12303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:03.872231 12109 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 06:05:03.872339 12109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-1/wal
uuid: "81fbba8a10f54078bade0f78cc102007"
format_stamp: "Formatted at 2026-08-08 06:05:03 on dist-test-slave-zj8q"
server_key: "3b8e708897585a0ed32cc8cf8842eae0"
server_key_iv: "c1537ecf126d40547e60c335c945b340"
server_key_version: "encryptionkey@0"
I20260808 06:05:03.872478 12109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:03.892560 12109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:03.894769 12109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:03.902415 12109 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.189:44883
I20260808 06:05:03.902534 12397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.189:44883 every 8 connection(s)
I20260808 06:05:03.903153 12109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-1/data/info.pb
I20260808 06:05:03.905954 12398 sys_catalog.cc:263] Verifying existing consensus state
I20260808 06:05:03.907265 12398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007: Bootstrap starting.
I20260808 06:05:03.917924 12398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 06:05:03.918233 12398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007: Bootstrap complete.
I20260808 06:05:03.918797 12398 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: NON_VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: true } }
I20260808 06:05:03.918942 12398 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 81fbba8a10f54078bade0f78cc102007, State: Initialized, Role: LEARNER
I20260808 06:05:03.919054 12398 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: NON_VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: true } }
I20260808 06:05:03.919405 12401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: NON_VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: true } } }
I20260808 06:05:03.919518 12398 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:05:03.919615 12401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [sys.catalog]: This master's current role is: LEARNER
I20260808 06:05:03.922830 12419 catalog_manager.cc:1259] Loaded cluster ID: 34f12aa373e04701a013720e69244e25
I20260808 06:05:03.922885 12419 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007: loading cluster ID for follower catalog manager: success
I20260808 06:05:03.924862 12419 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007: acquiring CA information for follower catalog manager: success
I20260808 06:05:03.926195 12419 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 06:05:03.991869 12285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:03.992111 12285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:03.992169 12285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:03.996199 12285 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:05:03.996253 12285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:03.996280 12285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:03.996302 12285 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:05:03.996331 12285 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:05:04.001503 12285 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:45251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:35293,127.1.151.189:44883,127.1.151.188:33465
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.188.keytab
--principal=oryx/127.1.151.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.188:33465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.151.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:05:04.002573 12285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:04.003573 12285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:04.012090 12427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:04.012102 12429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:04.012146 12425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:04.012323 12285 server_base.cc:1034] running on GCE node
I20260808 06:05:04.013118 12285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:04.014482 12285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:04.015673 12285 hybrid_clock.cc:648] HybridClock initialized: now 1786169104015646 us; error 34 us; skew 500 ppm
Aug 08 06:05:04 dist-test-slave-zj8q krb5kdc[11323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169104, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:04.019544 12285 init.cc:377] Logged in from keytab as oryx/127.1.151.188@KRBTEST.COM (short username oryx)
I20260808 06:05:04.020422 12285 webserver.cc:460] Webserver started at http://127.1.151.188:44621/ using document root <none> and password file <none>
I20260808 06:05:04.020974 12285 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:04.021042 12285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:04.021248 12285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:05:04.023307 12285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-2/data/instance:
uuid: "4275957a7be24a9cb4d20a46934a9d1f"
format_stamp: "Formatted at 2026-08-08 06:05:04 on dist-test-slave-zj8q"
server_key: "51efb1555320f7536806bab854bf20ac"
server_key_iv: "71db909c764a96ea3b96875631e44801"
server_key_version: "encryptionkey@0"
I20260808 06:05:04.023875 12285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-2/wal/instance:
uuid: "4275957a7be24a9cb4d20a46934a9d1f"
format_stamp: "Formatted at 2026-08-08 06:05:04 on dist-test-slave-zj8q"
server_key: "51efb1555320f7536806bab854bf20ac"
server_key_iv: "71db909c764a96ea3b96875631e44801"
server_key_version: "encryptionkey@0"
I20260808 06:05:04.027525 12285 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 06:05:04.029798 12438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:04.030983 12285 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 06:05:04.031123 12285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-2/wal
uuid: "4275957a7be24a9cb4d20a46934a9d1f"
format_stamp: "Formatted at 2026-08-08 06:05:04 on dist-test-slave-zj8q"
server_key: "51efb1555320f7536806bab854bf20ac"
server_key_iv: "71db909c764a96ea3b96875631e44801"
server_key_version: "encryptionkey@0"
I20260808 06:05:04.031210 12285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:04.046353 12285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:04.049127 12285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:05:04.049304 12285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:04.060048 12285 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.188:33465
I20260808 06:05:04.060431 12517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.188:33465 every 8 connection(s)
I20260808 06:05:04.061007 12285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-2/data/info.pb
I20260808 06:05:04.062923 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 12285
I20260808 06:05:04.065016 12519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:05:04.069432 12519 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 06:05:04 dist-test-slave-zj8q krb5kdc[11323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.151.188@KRBTEST.COM
Aug 08 06:05:04 dist-test-slave-zj8q krb5kdc[11323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169104, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.188@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM
I20260808 06:05:04.077955 12522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.064417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41038 (local address 127.1.151.188:33465)
0808 06:05:04.064850 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:04.064865 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:04.064899 (+ 34us) server_negotiation.cc:407] Connection header received
0808 06:05:04.065817 (+ 918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:04.065836 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:04.066176 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:04.066490 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:04.067293 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.068600 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.069478 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.070019 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.070208 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.072937 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:04.072943 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:04.072955 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:04.072985 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:04.075027 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.075491 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.075497 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.075503 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.075616 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.075869 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.075873 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.075874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.076239 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:04.076421 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:04.076841 (+ 420us) server_negotiation.cc:299] Negotiation successful
0808 06:05:04.077132 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":151,"threads_started":1}
I20260808 06:05:04.080621 11447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.071307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57812 (local address 127.1.151.190:35293)
0808 06:05:04.071438 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:04.071442 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:04.071642 (+ 200us) server_negotiation.cc:407] Connection header received
0808 06:05:04.071788 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:04.071791 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:04.071839 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:04.071925 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:04.072709 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.073456 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.074593 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.074930 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.075039 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.077560 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:04.077563 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:04.077565 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:04.077590 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:04.079001 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.079397 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.079400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.079402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.079493 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.079776 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.079778 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.079780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.080011 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:04.080109 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:04.080414 (+ 305us) server_negotiation.cc:299] Negotiation successful
0808 06:05:04.080509 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 06:05:04.080636 12523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.071091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35293 (local address 127.0.0.1:57812)
0808 06:05:04.071508 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:04.071562 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:04.071699 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:04.071978 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:04.071983 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:04.072344 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:04.072556 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.072563 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.073620 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.073624 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.074472 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.074483 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.075036 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.075039 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.075176 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.075692 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:04.075704 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:04.077426 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:04.079095 (+ 1669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:04.079102 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:04.079107 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:04.079306 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:04.079584 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:04.079586 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:04.079588 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:04.079694 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:04.080145 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:04.080151 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:04.080248 (+ 97us) client_negotiation.cc:769] Sending connection context
0808 06:05:04.080356 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 06:05:04.080481 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":98,"threads_started":1}
I20260808 06:05:04.081629 12519 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 06:05:04 dist-test-slave-zj8q krb5kdc[11323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169104, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.188@KRBTEST.COM for oryx/127.1.151.189@KRBTEST.COM
I20260808 06:05:04.089051 12523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.082093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.189:44883 (local address 127.0.0.1:59566)
0808 06:05:04.082222 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:04.082235 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:04.082295 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:04.082616 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:04.082618 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:04.082755 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:04.082908 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.082914 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.083739 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.083741 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.084244 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.084250 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.084593 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.084595 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.084679 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.085006 (+ 327us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:04.085016 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:04.086135 (+ 1119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:04.087737 (+ 1602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:04.087740 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:04.087742 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:04.087916 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:04.088158 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:04.088161 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:04.088163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:04.088237 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:04.088565 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:04.088568 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:04.088656 (+ 88us) client_negotiation.cc:769] Sending connection context
0808 06:05:04.088742 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 06:05:04.088894 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 06:05:04.089066 12524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.082220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59566 (local address 127.1.151.189:44883)
0808 06:05:04.082440 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:04.082443 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:04.082454 (+ 11us) server_negotiation.cc:407] Connection header received
0808 06:05:04.082490 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:04.082492 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:04.082530 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:04.082632 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:04.082997 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.083661 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.084319 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.084524 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.084589 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.086289 (+ 1700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:04.086292 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:04.086294 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:04.086316 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:04.087637 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.088001 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.088004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.088005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.088077 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.088313 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.088316 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.088317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.088478 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:04.088560 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:04.088807 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 06:05:04.088916 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":78,"threads_started":1}
I20260808 06:05:04.089854 12519 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.151.188" port: 33465 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 06:05:04 dist-test-slave-zj8q krb5kdc[11323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169104, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.188@KRBTEST.COM for oryx/127.1.151.188@KRBTEST.COM
I20260808 06:05:04.097069 12523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.090264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:33465 (local address 127.0.0.1:41044)
0808 06:05:04.090366 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:04.090378 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:04.090453 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:04.090631 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:04.090633 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:04.090769 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:04.090827 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.090832 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.091988 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.091991 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.092514 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.092522 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.092958 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.092962 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.093032 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.093346 (+ 314us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:04.093356 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:04.094451 (+ 1095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:04.095829 (+ 1378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:04.095832 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:04.095833 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:04.096010 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:04.096263 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:04.096266 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:04.096267 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:04.096348 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:04.096664 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:04.096666 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:04.096745 (+ 79us) client_negotiation.cc:769] Sending connection context
0808 06:05:04.096826 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 06:05:04.096924 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 06:05:04.097090 12522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.090301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41044 (local address 127.1.151.188:33465)
0808 06:05:04.090385 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:04.090389 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:04.090418 (+ 29us) server_negotiation.cc:407] Connection header received
0808 06:05:04.090515 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:04.090518 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:04.090550 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:04.090617 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:04.090920 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.091389 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.092628 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.092870 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.092950 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.094578 (+ 1628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:04.094580 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:04.094583 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:04.094624 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:04.095732 (+ 1108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.096107 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.096110 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.096111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.096181 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.096423 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.096425 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.096426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.096572 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:04.096651 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:04.096866 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 06:05:04.096976 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":34}
I20260808 06:05:04.100175 12519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f: Bootstrap starting.
I20260808 06:05:04.102674 12519 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f: Neither blocks nor log segments found. Creating new log.
I20260808 06:05:04.103838 12519 log.cc:824] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f: Log is configured to *not* fsync() on all Append() calls
I20260808 06:05:04.106014 12519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f: No bootstrap required, opened a new log
I20260808 06:05:04.109956 12519 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } } peers { permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 33465 } }
I20260808 06:05:04.110174 12519 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:05:04.110245 12519 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4275957a7be24a9cb4d20a46934a9d1f, State: Initialized, Role: FOLLOWER
I20260808 06:05:04.110616 12519 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } } peers { permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 33465 } }
I20260808 06:05:04.111204 12526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } } peers { permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 33465 } } }
I20260808 06:05:04.111367 12526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 06:05:04.111831 12519 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [sys.catalog]: configured and running, proceeding with master startup.
W20260808 06:05:04.117445 12543 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 06:05:04.117516 12543 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 06:05:04.120944 12285 master_runner.cc:428] Detected that this master 4275957a7be24a9cb4d20a46934a9d1f is joining an existing cluster
I20260808 06:05:04.120992 12285 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.151.188:33465
I20260808 06:05:04.125974 11682 heartbeater.cc:503] Master 127.1.151.190:35293 was elected leader, sending a full tablet report...
I20260808 06:05:04.131712 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 11475
I20260808 06:05:04.133028 12554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.123183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35293 (local address 127.0.0.1:57824)
0808 06:05:04.123490 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:04.123505 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:04.123620 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:04.124025 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:04.124028 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:04.124408 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:04.125051 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.125062 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.126080 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.126083 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.127257 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.127269 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.127927 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.127931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.128023 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.129088 (+ 1065us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:04.129103 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:04.129713 (+ 610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:04.131610 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:04.131613 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:04.131615 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:04.131898 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:04.132182 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:04.132185 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:04.132187 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:04.132261 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:04.132630 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:04.132633 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:04.132708 (+ 75us) client_negotiation.cc:769] Sending connection context
0808 06:05:04.132787 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 06:05:04.132896 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":25,"thread_start_us":90,"threads_started":1}
I20260808 06:05:04.133380 11447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.123289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57824 (local address 127.1.151.190:35293)
0808 06:05:04.123420 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:04.123424 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:04.123579 (+ 155us) server_negotiation.cc:407] Connection header received
0808 06:05:04.123751 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:04.123755 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:04.123809 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:04.123866 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:04.125176 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.125940 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.127370 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.127689 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.127795 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.129833 (+ 2038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:04.129836 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:04.129839 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:04.129864 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:04.131463 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.132003 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.132007 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.132008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.132088 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.132339 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.132342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.132343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.132513 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:04.132592 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:04.133186 (+ 594us) server_negotiation.cc:299] Negotiation successful
0808 06:05:04.133275 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 06:05:04.162691 12522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.124513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41052 (local address 127.1.151.188:33465)
0808 06:05:04.124608 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:04.124612 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:04.125403 (+ 791us) server_negotiation.cc:407] Connection header received
0808 06:05:04.125544 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:04.125547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:04.125587 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:04.125627 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:04.127815 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.128449 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.129644 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.129970 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.130077 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.157420 (+ 27343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:04.157424 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:04.157427 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:04.157457 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:04.159101 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.161439 (+ 2338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.161443 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.161445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.161537 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.161806 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.161809 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.161811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.162018 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:04.162088 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:04.162390 (+ 302us) server_negotiation.cc:299] Negotiation successful
0808 06:05:04.162508 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 06:05:04.163208 12556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.124371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:33465 (local address 127.0.0.1:41052)
0808 06:05:04.125299 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:04.125311 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:04.125453 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:04.125725 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:04.125727 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:04.127546 (+ 1819us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:04.127714 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.127722 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.128585 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.128589 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.129504 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.129516 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.130594 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.130604 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.130847 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.131531 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:04.131545 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:04.157215 (+ 25670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:04.159964 (+ 2749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:04.159967 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:04.159969 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:04.160579 (+ 610us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:04.161641 (+ 1062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:04.161644 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:04.161646 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:04.161725 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:04.162143 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:04.162146 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:04.162247 (+ 101us) client_negotiation.cc:769] Sending connection context
0808 06:05:04.162888 (+ 641us) client_negotiation.cc:240] Negotiation successful
0808 06:05:04.163041 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":866,"thread_start_us":513,"threads_started":1}
I20260808 06:05:04.163892 12524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.123777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59572 (local address 127.1.151.189:44883)
0808 06:05:04.123892 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:04.123896 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:04.124315 (+ 419us) server_negotiation.cc:407] Connection header received
0808 06:05:04.124522 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:04.124525 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:04.124577 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:04.124656 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:04.125328 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.126147 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.127667 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.128022 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.128176 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.159637 (+ 31461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:04.159641 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:04.159644 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:04.159676 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:04.161230 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.161787 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.161791 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.161793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.161883 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.162451 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.162455 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.162457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.162694 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:04.162969 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:04.163603 (+ 634us) server_negotiation.cc:299] Negotiation successful
0808 06:05:04.163749 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 06:05:04.164453 12555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.123727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.189:44883 (local address 127.0.0.1:59572)
0808 06:05:04.124227 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:04.124242 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:04.124386 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:04.124693 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:04.124695 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:04.125025 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:04.125186 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.125195 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.126337 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.126340 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.127492 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.127500 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.128341 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.128345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.128459 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.157905 (+ 29446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:04.157920 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:04.159449 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:04.161358 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:04.161361 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:04.161363 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:04.161663 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:04.161987 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:04.161990 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:04.161992 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:04.162112 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:04.163052 (+ 940us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:04.163056 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:04.163258 (+ 202us) client_negotiation.cc:769] Sending connection context
0808 06:05:04.164098 (+ 840us) client_negotiation.cc:240] Negotiation successful
0808 06:05:04.164235 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"mutex_wait_us":1,"spinlock_wait_cycles":2816,"thread_start_us":257,"threads_started":1}
I20260808 06:05:04.168313 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.129:38287
--local_ip_for_outbound_sockets=127.1.151.129
--tserver_master_addrs=127.1.151.190:35293
--webserver_port=41815
--webserver_interface=127.1.151.129
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.129.keytab
--principal=oryx/127.1.151.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:45251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.151.190:35293,127.1.151.189:44883
--tserver_master_addrs=127.1.151.190:35293,127.1.151.189:44883,127.1.151.188:33465 with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 06:05:04 dist-test-slave-zj8q krb5kdc[11323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169102, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.190@KRBTEST.COM for oryx/127.1.151.188@KRBTEST.COM
I20260808 06:05:04.173404 12266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.164963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:33465 (local address 127.0.0.1:41062)
0808 06:05:04.165394 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:04.165409 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:04.165491 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:04.165808 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:04.165811 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:04.165980 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:04.166069 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.166076 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.166794 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.166798 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.167341 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.167349 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.167831 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.167834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.167934 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.168395 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:04.168409 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:04.169934 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:04.171725 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:04.171729 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:04.171731 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:04.171981 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:04.172255 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:04.172258 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:04.172260 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:04.172356 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:04.172833 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:04.172836 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:04.172946 (+ 110us) client_negotiation.cc:769] Sending connection context
0808 06:05:04.173055 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 06:05:04.173228 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345}
I20260808 06:05:04.174093 12522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.165187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41062 (local address 127.1.151.188:33465)
0808 06:05:04.165300 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:04.165304 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:04.165586 (+ 282us) server_negotiation.cc:407] Connection header received
0808 06:05:04.165652 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:04.165655 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:04.165701 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:04.165765 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:04.166209 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.166692 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.167446 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.167726 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.168393 (+ 667us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.170090 (+ 1697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:04.170094 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:04.170097 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:04.170122 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:04.171599 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.172074 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.172078 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.172080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.172167 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.172440 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.172443 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.172445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.172730 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:04.172821 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:04.173813 (+ 992us) server_negotiation.cc:299] Negotiation successful
0808 06:05:04.173932 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 06:05:04.175351 11384 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.151.188:33465: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f"
member_type: NON_VOTER
last_known_addr {
host: "127.1.151.188"
port: 33465
}
attrs {
promote: true
}
}
dest_uuid: "88cc958608974ee987b790f318d04d52"
cas_config_opid_index: 5
I20260808 06:05:04.176014 11384 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: NON_VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: true } } peers { permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: NON_VOTER last_known_addr { host: "127.1.151.188" port: 33465 } attrs { promote: true } }
I20260808 06:05:04.177234 12270 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 4275957a7be24a9cb4d20a46934a9d1f (127.1.151.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: NON_VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: true } } peers { permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: NON_VOTER last_known_addr { host: "127.1.151.188" port: 33465 } attrs { promote: true } } }
I20260808 06:05:04.178112 12481 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:05:04.179009 12270 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.151.188:33465
I20260808 06:05:04.179208 12270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "88cc958608974ee987b790f318d04d52" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: NON_VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: true } } peers { permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: NON_VOTER last_known_addr { host: "127.1.151.188" port: 33465 } attrs { promote: true } } }
I20260808 06:05:04.179308 12270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [sys.catalog]: This master's current role is: LEADER
I20260808 06:05:04.179582 12481 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [term 1 FOLLOWER]: Refusing update from remote peer 88cc958608974ee987b790f318d04d52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 06:05:04.180186 12269 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: NON_VOTER last_known_addr { host: "127.1.151.188" port: 33465 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:05:04.180276 12285 master.cc:537] Master@127.1.151.188:33465 shutting down...
I20260808 06:05:04.185467 12266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.177276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.189:44883 (local address 127.0.0.1:59580)
0808 06:05:04.180786 (+ 3510us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:04.180802 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:04.180942 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:04.181417 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:04.181420 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:04.181439 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:05:04.181523 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.181529 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.182634 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.182638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.183915 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.183926 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.185056 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.185059 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.185167 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.185210 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 06:05:04.185276 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 06:05:04.185344 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3424}
I20260808 06:05:04.185918 12524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.177509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59580 (local address 127.1.151.189:44883)
0808 06:05:04.180017 (+ 2508us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:04.180023 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:04.180863 (+ 840us) server_negotiation.cc:407] Connection header received
0808 06:05:04.181088 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:04.181092 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:04.181165 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:04.181244 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:05:04.181647 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.182495 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.184059 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.184734 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.184811 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.184962 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:04.185723 (+ 761us) server_negotiation.cc:299] Negotiation successful
0808 06:05:04.185802 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2408}
W20260808 06:05:04.185817 11364 consensus_peers.cc:487] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 -> Peer 4275957a7be24a9cb4d20a46934a9d1f (127.1.151.188:33465): Couldn't send request to peer 4275957a7be24a9cb4d20a46934a9d1f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 06:05:04.186695 12350 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [term 1 LEARNER]: Refusing update from remote peer 88cc958608974ee987b790f318d04d52: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 06:05:04.187112 12269 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: NON_VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 06:05:04.187879 12285 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:05:04.188272 12285 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:05:04.188374 12285 tablet_replica.cc:330] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f: stopping tablet replica
I20260808 06:05:04.189080 12350 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 4275957a7be24a9cb4d20a46934a9d1f (127.1.151.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: NON_VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: true } } peers { permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: NON_VOTER last_known_addr { host: "127.1.151.188" port: 33465 } attrs { promote: true } } }
I20260808 06:05:04.191416 12401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88cc958608974ee987b790f318d04d52. Latest consensus state: current_term: 1 leader_uuid: "88cc958608974ee987b790f318d04d52" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: NON_VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: true } } peers { permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: NON_VOTER last_known_addr { host: "127.1.151.188" port: 33465 } attrs { promote: true } } }
I20260808 06:05:04.191534 12401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [sys.catalog]: This master's current role is: LEARNER
I20260808 06:05:04.191630 12401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "88cc958608974ee987b790f318d04d52" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: NON_VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: true } } peers { permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: NON_VOTER last_known_addr { host: "127.1.151.188" port: 33465 } attrs { promote: true } } }
I20260808 06:05:04.191700 12401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [sys.catalog]: This master's current role is: LEARNER
I20260808 06:05:04.193363 12285 master.cc:552] Master@127.1.151.188:33465 shutdown complete.
I20260808 06:05:04.193413 12285 master_runner.cc:305] Clearing existing system tablet
I20260808 06:05:04.193656 12285 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:04.193708 12285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:04.194983 12285 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 06:05:04.195851 12564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:04.196101 12285 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:04.196223 12285 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 06:05:04.196260 12285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-2/wal
uuid: "4275957a7be24a9cb4d20a46934a9d1f"
format_stamp: "Formatted at 2026-08-08 06:05:04 on dist-test-slave-zj8q"
server_key: "51efb1555320f7536806bab854bf20ac"
server_key_iv: "71db909c764a96ea3b96875631e44801"
server_key_version: "encryptionkey@0"
I20260808 06:05:04.198333 12285 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:05:04.200572 12285 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 06:05:04.200671 12285 log.cc:1196] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 06:05:04.200904 12285 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f: Deleting consensus metadata
I20260808 06:05:04.201058 12285 master_runner.cc:315] Copying system tablet from 127.1.151.190:35293
I20260808 06:05:04.202128 12285 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f: tablet copy: Beginning tablet copy session from remote peer at address 127.1.151.190:35293
I20260808 06:05:04.202421 12269 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52: attempting to promote NON_VOTER 81fbba8a10f54078bade0f78cc102007 to VOTER
I20260808 06:05:04.202939 12269 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: false } } peers { permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: NON_VOTER last_known_addr { host: "127.1.151.188" port: 33465 } attrs { promote: true } }
I20260808 06:05:04.204336 12266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.204028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:33465 (local address 127.0.0.1:41068)
0808 06:05:04.204167 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:04.204248 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:33465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 06:05:04.204852 12350 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [term 1 LEARNER]: Refusing update from remote peer 88cc958608974ee987b790f318d04d52: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 06:05:04.205432 12266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.205048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:33465 (local address 127.0.0.1:41084)
0808 06:05:04.205317 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:04.205369 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:33465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203}
I20260808 06:05:04.205583 12270 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260808 06:05:04.205855 11364 consensus_peers.cc:487] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 -> Peer 4275957a7be24a9cb4d20a46934a9d1f (127.1.151.188:33465): Couldn't send request to peer 4275957a7be24a9cb4d20a46934a9d1f. Status: Network error: Client connection negotiation failed: client connection to 127.1.151.188:33465: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 06:05:04.207384 12270 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 81fbba8a10f54078bade0f78cc102007 (127.1.151.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: false } } peers { permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: NON_VOTER last_known_addr { host: "127.1.151.188" port: 33465 } attrs { promote: true } } }
I20260808 06:05:04.207886 12350 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 81fbba8a10f54078bade0f78cc102007 (127.1.151.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: false } } peers { permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: NON_VOTER last_known_addr { host: "127.1.151.188" port: 33465 } attrs { promote: true } } }
I20260808 06:05:04.208843 12270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "88cc958608974ee987b790f318d04d52" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: false } } peers { permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: NON_VOTER last_known_addr { host: "127.1.151.188" port: 33465 } attrs { promote: true } } }
I20260808 06:05:04.208940 12270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [sys.catalog]: This master's current role is: LEADER
I20260808 06:05:04.209048 12270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "88cc958608974ee987b790f318d04d52" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: false } } peers { permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: NON_VOTER last_known_addr { host: "127.1.151.188" port: 33465 } attrs { promote: true } } }
I20260808 06:05:04.209113 12270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [sys.catalog]: This master's current role is: LEADER
I20260808 06:05:04.209599 12562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "88cc958608974ee987b790f318d04d52" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: false } } peers { permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: NON_VOTER last_known_addr { host: "127.1.151.188" port: 33465 } attrs { promote: true } } }
I20260808 06:05:04.209725 12562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 06:05:04.218860 11447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.206541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57832 (local address 127.1.151.190:35293)
0808 06:05:04.206641 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:04.206645 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:04.210591 (+ 3946us) server_negotiation.cc:407] Connection header received
0808 06:05:04.210707 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:04.210710 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:04.210759 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:04.210840 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:04.211699 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.212501 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.213682 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.214045 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.214139 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.215408 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:04.215412 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:04.215415 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:04.215441 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:04.216986 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.217529 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.217532 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.217534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.217631 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.217903 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.217906 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.217908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.218110 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:04.218315 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:04.218592 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 06:05:04.218712 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 06:05:04.218860 12578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.207157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35293 (local address 127.0.0.1:57832)
0808 06:05:04.210491 (+ 3334us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:04.210506 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:04.210624 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:04.210871 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:04.210873 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:04.211220 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:04.211560 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.211572 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.212616 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.212620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.213565 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.213578 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.214173 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.214178 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.214276 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.214820 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:04.214835 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:04.215263 (+ 428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:04.217183 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:04.217186 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:04.217188 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:04.217433 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:04.217719 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:04.217722 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:04.217724 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:04.217817 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:04.218315 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:04.218318 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:04.218422 (+ 104us) client_negotiation.cc:769] Sending connection context
0808 06:05:04.218531 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 06:05:04.218669 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3256,"thread_start_us":3169,"threads_started":1}
I20260808 06:05:04.219533 11413 tablet_copy_service.cc:140] P 88cc958608974ee987b790f318d04d52: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 4275957a7be24a9cb4d20a46934a9d1f ({username='oryx', principal='oryx/127.1.151.188@KRBTEST.COM'} at 127.0.0.1:57832)
I20260808 06:05:04.219631 11413 tablet_copy_service.cc:161] P 88cc958608974ee987b790f318d04d52: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 4275957a7be24a9cb4d20a46934a9d1f at {username='oryx', principal='oryx/127.1.151.188@KRBTEST.COM'} at 127.0.0.1:57832: session id = 4275957a7be24a9cb4d20a46934a9d1f-00000000000000000000000000000000
I20260808 06:05:04.220665 11413 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 06:05:04.221977 12285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 06:05:04.224227 12285 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f: tablet copy: Starting download of 0 data blocks...
I20260808 06:05:04.224407 12285 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f: tablet copy: Starting download of 1 WAL segments...
I20260808 06:05:04.225893 12285 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 06:05:04.227684 11413 tablet_copy_service.cc:342] P 88cc958608974ee987b790f318d04d52: Request end of tablet copy session 4275957a7be24a9cb4d20a46934a9d1f-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.151.188@KRBTEST.COM'} at 127.0.0.1:57832
I20260808 06:05:04.227763 11413 tablet_copy_service.cc:433] P 88cc958608974ee987b790f318d04d52: ending tablet copy session 4275957a7be24a9cb4d20a46934a9d1f-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 4275957a7be24a9cb4d20a46934a9d1f
I20260808 06:05:04.234351 12285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:04.235880 12579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:04.236057 12580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:04.236124 12582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:04.236560 12285 server_base.cc:1034] running on GCE node
I20260808 06:05:04.236721 12285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:04.237010 12285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:04.238193 12285 hybrid_clock.cc:648] HybridClock initialized: now 1786169104238181 us; error 31 us; skew 500 ppm
Aug 08 06:05:04 dist-test-slave-zj8q krb5kdc[11323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169104, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:04.239714 12285 init.cc:377] Logged in from keytab as oryx/127.1.151.188@KRBTEST.COM (short username oryx)
I20260808 06:05:04.240132 12285 webserver.cc:460] Webserver started at http://127.1.151.188:44411/ using document root <none> and password file <none>
I20260808 06:05:04.240306 12285 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:04.240345 12285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:04.241554 12285 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 06:05:04.242530 12594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:04.242754 12285 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 06:05:04.242816 12285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-2/wal
uuid: "4275957a7be24a9cb4d20a46934a9d1f"
format_stamp: "Formatted at 2026-08-08 06:05:04 on dist-test-slave-zj8q"
server_key: "51efb1555320f7536806bab854bf20ac"
server_key_iv: "71db909c764a96ea3b96875631e44801"
server_key_version: "encryptionkey@0"
I20260808 06:05:04.242894 12285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:04.270588 12285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:04.272930 12285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:04.278110 12285 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.188:33465
I20260808 06:05:04.278141 12677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.188:33465 every 8 connection(s)
I20260808 06:05:04.278739 12285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/master-2/data/info.pb
I20260808 06:05:04.280592 12678 sys_catalog.cc:263] Verifying existing consensus state
I20260808 06:05:04.281648 12678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f: Bootstrap starting.
I20260808 06:05:04.293677 12678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 06:05:04.293967 12678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f: Bootstrap complete.
I20260808 06:05:04.294783 12678 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: false } } peers { permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: NON_VOTER last_known_addr { host: "127.1.151.188" port: 33465 } attrs { promote: true } }
I20260808 06:05:04.294957 12678 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4275957a7be24a9cb4d20a46934a9d1f, State: Initialized, Role: LEARNER
I20260808 06:05:04.295094 12678 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: false } } peers { permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: NON_VOTER last_known_addr { host: "127.1.151.188" port: 33465 } attrs { promote: true } }
I20260808 06:05:04.295457 12683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: false } } peers { permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: NON_VOTER last_known_addr { host: "127.1.151.188" port: 33465 } attrs { promote: true } } }
I20260808 06:05:04.295681 12683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [sys.catalog]: This master's current role is: LEARNER
I20260808 06:05:04.295569 12678 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:05:04.298740 12700 catalog_manager.cc:1259] Loaded cluster ID: 34f12aa373e04701a013720e69244e25
I20260808 06:05:04.298851 12700 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f: loading cluster ID for follower catalog manager: success
I20260808 06:05:04.301077 12700 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f: acquiring CA information for follower catalog manager: success
I20260808 06:05:04.302595 12700 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 06:05:04.317601 12558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:04.317862 12558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:04.317927 12558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:04.322925 12558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:04.323010 12558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:04.323105 12558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.129
I20260808 06:05:04.328996 12558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:45251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.129.keytab
--principal=oryx/127.1.151.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.129:38287
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.151.129
--webserver_port=41815
--tserver_master_addrs=127.1.151.190:35293,127.1.151.189:44883,127.1.151.188:33465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.129
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:05:04.329957 12558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:04.330928 12558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:04.340631 12707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:04.340715 12705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:04.340574 12709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:04.340910 12558 server_base.cc:1034] running on GCE node
I20260808 06:05:04.341254 12558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:04.341847 12558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:04.342523 11889 heartbeater.cc:503] Master 127.1.151.190:35293 was elected leader, sending a full tablet report...
I20260808 06:05:04.343034 12558 hybrid_clock.cc:648] HybridClock initialized: now 1786169104343000 us; error 48 us; skew 500 ppm
Aug 08 06:05:04 dist-test-slave-zj8q krb5kdc[11323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169104, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:04.346617 12558 init.cc:377] Logged in from keytab as oryx/127.1.151.129@KRBTEST.COM (short username oryx)
I20260808 06:05:04.347448 12558 webserver.cc:460] Webserver started at http://127.1.151.129:41815/ using document root <none> and password file <none>
I20260808 06:05:04.347990 12558 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:04.348074 12558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:04.352149 12558 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 06:05:04.354357 12719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:04.355592 12558 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:05:04.355707 12558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-0/wal
uuid: "a67dd2b3f83143dd958e5cb489a649c2"
format_stamp: "Formatted at 2026-08-08 06:05:03 on dist-test-slave-zj8q"
server_key: "e175e1031493194e83dbb9689a81e85b"
server_key_iv: "0aa7960da8ffdc86bcc1e20442539200"
server_key_version: "encryptionkey@0"
I20260808 06:05:04.356348 12558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:04.370656 12558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:04.373845 12558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:05:04.374080 12558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:04.374789 12558 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:05:04.375702 12558 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:05:04.375748 12558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:04.375802 12558 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:05:04.375829 12558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:04.390712 12558 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.129:38287
I20260808 06:05:04.390749 12888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.129:38287 every 8 connection(s)
I20260808 06:05:04.391748 12558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
I20260808 06:05:04.392916 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 12558
I20260808 06:05:04.393103 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 11690
Aug 08 06:05:04 dist-test-slave-zj8q krb5kdc[11323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169104, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.189@KRBTEST.COM
I20260808 06:05:04.405898 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.130:38501
--local_ip_for_outbound_sockets=127.1.151.130
--tserver_master_addrs=127.1.151.190:35293
--webserver_port=46883
--webserver_interface=127.1.151.130
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.130.keytab
--principal=oryx/127.1.151.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:45251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.151.190:35293,127.1.151.189:44883
--tserver_master_addrs=127.1.151.190:35293,127.1.151.189:44883,127.1.151.188:33465 with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 06:05:04 dist-test-slave-zj8q krb5kdc[11323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169104, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.188@KRBTEST.COM
Aug 08 06:05:04 dist-test-slave-zj8q krb5kdc[11323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169104, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM
I20260808 06:05:04.411391 12524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.396852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:49549 (local address 127.1.151.189:44883)
0808 06:05:04.398588 (+ 1736us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:04.398593 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:04.398608 (+ 15us) server_negotiation.cc:407] Connection header received
0808 06:05:04.398655 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:04.398658 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:04.398704 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:04.398784 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:04.400462 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.401271 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.402780 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.403118 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.403222 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.406542 (+ 3320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:04.406545 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:04.406547 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:04.406571 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:04.408536 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.409369 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.409373 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.409375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.409465 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.410105 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.410109 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.410111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.410361 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:04.410430 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:04.411089 (+ 659us) server_negotiation.cc:299] Negotiation successful
0808 06:05:04.411217 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1644}
I20260808 06:05:04.411937 12899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.397094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:59847 (local address 127.1.151.188:33465)
0808 06:05:04.398594 (+ 1500us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:04.398645 (+ 51us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:04.398662 (+ 17us) server_negotiation.cc:407] Connection header received
0808 06:05:04.398707 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:04.398710 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:04.398760 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:04.398851 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:04.401425 (+ 2574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.402545 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.404064 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.404345 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.404718 (+ 373us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.407741 (+ 3023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:04.407745 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:04.407747 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:04.407778 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:04.409961 (+ 2183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.410460 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.410464 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.410466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.410557 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.410863 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.410867 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.410868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.411088 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:04.411176 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:04.411652 (+ 476us) server_negotiation.cc:299] Negotiation successful
0808 06:05:04.411779 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1406,"thread_start_us":1324,"threads_started":1}
I20260808 06:05:04.413398 12896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.394800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:33465 (local address 127.1.151.129:59847)
0808 06:05:04.395865 (+ 1065us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:04.395916 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:04.397255 (+ 1339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:04.399056 (+ 1801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:04.399065 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:04.399557 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:04.400289 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.400309 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.402705 (+ 2396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.402708 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.403920 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.403932 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.404829 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.404833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.404979 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.406008 (+ 1029us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:04.406023 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:04.407573 (+ 1550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:04.410074 (+ 2501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:04.410077 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:04.410079 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:04.410347 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:04.410667 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:04.410669 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:04.410671 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:04.410759 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:04.411237 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:04.411239 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:04.411352 (+ 113us) client_negotiation.cc:769] Sending connection context
0808 06:05:04.412132 (+ 780us) client_negotiation.cc:240] Negotiation successful
0808 06:05:04.412303 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":848,"mutex_wait_us":40,"spinlock_wait_cycles":6016,"thread_start_us":652,"threads_started":1}
I20260808 06:05:04.413398 12897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.396337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.189:44883 (local address 127.1.151.129:49549)
0808 06:05:04.396714 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:04.396729 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:04.397258 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:04.399650 (+ 2392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:04.399652 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:04.400024 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:04.400289 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.400309 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.401482 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.401485 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.402636 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.402649 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.403474 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.403477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.403826 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.404443 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:04.404462 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:04.406385 (+ 1923us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:04.408907 (+ 2522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:04.408913 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:04.408927 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:04.409260 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:04.409817 (+ 557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:04.409820 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:04.409823 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:04.410004 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:04.410507 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:04.410513 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:04.410822 (+ 309us) client_negotiation.cc:769] Sending connection context
0808 06:05:04.411773 (+ 951us) client_negotiation.cc:240] Negotiation successful
0808 06:05:04.412368 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":166,"thread_start_us":145,"threads_started":1}
I20260808 06:05:04.415004 12889 heartbeater.cc:347] Connected to a master server at 127.1.151.188:33465
I20260808 06:05:04.415243 12889 heartbeater.cc:464] Registering TS with master...
I20260808 06:05:04.415416 12893 heartbeater.cc:347] Connected to a master server at 127.1.151.189:44883
I20260808 06:05:04.415520 12893 heartbeater.cc:464] Registering TS with master...
I20260808 06:05:04.415827 12889 heartbeater.cc:511] Master 127.1.151.188:33465 requested a full tablet report, sending...
I20260808 06:05:04.415834 12893 heartbeater.cc:511] Master 127.1.151.189:44883 requested a full tablet report, sending...
I20260808 06:05:04.416055 11447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.403439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:41269 (local address 127.1.151.190:35293)
0808 06:05:04.403558 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:04.403562 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:04.403871 (+ 309us) server_negotiation.cc:407] Connection header received
0808 06:05:04.404037 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:04.404040 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:04.404089 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:04.404418 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:04.406908 (+ 2490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.407651 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.408760 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.409100 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.409219 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.411620 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:04.411624 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:04.411627 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:04.411654 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:04.413435 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.414564 (+ 1129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.414568 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.414570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.414675 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.415000 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.415004 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.415006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.415257 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:04.415373 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:04.415779 (+ 406us) server_negotiation.cc:299] Negotiation successful
0808 06:05:04.415912 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 06:05:04.416769 12900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.402949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35293 (local address 127.1.151.129:41269)
0808 06:05:04.403753 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:04.403766 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:04.403937 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:04.404597 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:04.404600 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:04.405651 (+ 1051us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:04.406768 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.406781 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.407827 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.407829 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.408634 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.408646 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.409245 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.409248 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.409335 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.409888 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:04.409901 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:04.411433 (+ 1532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:04.413589 (+ 2156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:04.413592 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:04.413595 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:04.414442 (+ 847us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:04.414786 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:04.414789 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:04.414790 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:04.414877 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:04.415363 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:04.415366 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:04.415459 (+ 93us) client_negotiation.cc:769] Sending connection context
0808 06:05:04.416440 (+ 981us) client_negotiation.cc:240] Negotiation successful
0808 06:05:04.416580 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":736,"spinlock_wait_cycles":896,"thread_start_us":182,"threads_started":1}
I20260808 06:05:04.417523 12332 ts_manager.cc:194] Registered new tserver with Master: a67dd2b3f83143dd958e5cb489a649c2 (127.1.151.129:38287)
I20260808 06:05:04.417567 12613 ts_manager.cc:194] Registered new tserver with Master: a67dd2b3f83143dd958e5cb489a649c2 (127.1.151.129:38287)
I20260808 06:05:04.418773 12892 heartbeater.cc:347] Connected to a master server at 127.1.151.190:35293
I20260808 06:05:04.418876 12892 heartbeater.cc:464] Registering TS with master...
I20260808 06:05:04.419061 12892 heartbeater.cc:511] Master 127.1.151.190:35293 requested a full tablet report, sending...
I20260808 06:05:04.419823 11384 ts_manager.cc:194] Re-registered known tserver with Master: a67dd2b3f83143dd958e5cb489a649c2 (127.1.151.129:38287)
I20260808 06:05:04.420483 11384 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.151.129@KRBTEST.COM'} at 127.1.151.129:41269
W20260808 06:05:04.520179 12901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:04.520435 12901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:04.520493 12901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:04.524659 12901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:04.524711 12901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:04.524787 12901 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.130
I20260808 06:05:04.529762 12901 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:45251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.130.keytab
--principal=oryx/127.1.151.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.130:38501
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.151.130
--webserver_port=46883
--tserver_master_addrs=127.1.151.190:35293,127.1.151.189:44883,127.1.151.188:33465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.130
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:05:04.530709 12901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:04.531641 12901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:04.540796 12907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:04.540818 12901 server_base.cc:1034] running on GCE node
W20260808 06:05:04.540836 12911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:04.540813 12908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:04.541390 12901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:04.541463 12101 heartbeater.cc:503] Master 127.1.151.190:35293 was elected leader, sending a full tablet report...
I20260808 06:05:04.542050 12901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:04.543228 12901 hybrid_clock.cc:648] HybridClock initialized: now 1786169104543208 us; error 33 us; skew 500 ppm
Aug 08 06:05:04 dist-test-slave-zj8q krb5kdc[11323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169104, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:04.545955 12901 init.cc:377] Logged in from keytab as oryx/127.1.151.130@KRBTEST.COM (short username oryx)
I20260808 06:05:04.546797 12901 webserver.cc:460] Webserver started at http://127.1.151.130:46883/ using document root <none> and password file <none>
I20260808 06:05:04.547325 12901 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:04.547408 12901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:04.551127 12901 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260808 06:05:04.553202 12927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:04.554390 12901 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 06:05:04.554497 12901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-1/wal
uuid: "1f756da4d21243bfbc204b1f2950800b"
format_stamp: "Formatted at 2026-08-08 06:05:03 on dist-test-slave-zj8q"
server_key: "7413aca3209fb65c0355047c7119ecbb"
server_key_iv: "67d9aad79823dbceaa403091193b4fed"
server_key_version: "encryptionkey@0"
I20260808 06:05:04.555025 12901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:04.569113 12901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:04.571954 12901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:05:04.572127 12901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:04.572818 12901 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:05:04.573714 12901 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:05:04.573768 12901 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:04.573822 12901 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:05:04.573851 12901 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:04.582777 12901 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.130:38501
I20260808 06:05:04.582837 13085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.130:38501 every 8 connection(s)
I20260808 06:05:04.583820 12901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
I20260808 06:05:04.592895 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 12901
I20260808 06:05:04.593070 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 11900
Aug 08 06:05:04 dist-test-slave-zj8q krb5kdc[11323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169104, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.189@KRBTEST.COM
Aug 08 06:05:04 dist-test-slave-zj8q krb5kdc[11323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169104, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.188@KRBTEST.COM
I20260808 06:05:04.605171 12524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.587287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:49847 (local address 127.1.151.189:44883)
0808 06:05:04.587397 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:04.587400 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:04.587744 (+ 344us) server_negotiation.cc:407] Connection header received
0808 06:05:04.588061 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:04.588065 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:04.588114 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:04.588163 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:04.589625 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.590467 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.592086 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.592310 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.592399 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.601277 (+ 8878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:04.601281 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:04.601283 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:04.601310 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:04.602752 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.603397 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.603401 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.603403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.603507 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.603930 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.603933 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.603935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.604175 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:04.604279 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:04.604884 (+ 605us) server_negotiation.cc:299] Negotiation successful
0808 06:05:04.605001 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 06:05:04.606280 13094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.587173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.189:44883 (local address 127.1.151.130:49847)
0808 06:05:04.587637 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:04.587651 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:04.587812 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:04.588339 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:04.588347 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:04.588860 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:04.589513 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.589523 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.591336 (+ 1813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.591339 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.591970 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.591978 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.592446 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.592450 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.592634 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.599497 (+ 6863us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:04.599516 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:04.601087 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:04.602887 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:04.602894 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:04.602910 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:04.603244 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:04.603609 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:04.603612 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:04.603614 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:04.603822 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:04.604288 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:04.604295 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:04.604601 (+ 306us) client_negotiation.cc:769] Sending connection context
0808 06:05:04.604819 (+ 218us) client_negotiation.cc:240] Negotiation successful
0808 06:05:04.605164 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"mutex_wait_us":95,"spinlock_wait_cycles":10880,"thread_start_us":153,"threads_started":1}
I20260808 06:05:04.608062 13090 heartbeater.cc:347] Connected to a master server at 127.1.151.189:44883
I20260808 06:05:04.608347 13090 heartbeater.cc:464] Registering TS with master...
Aug 08 06:05:04 dist-test-slave-zj8q krb5kdc[11323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169104, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM
I20260808 06:05:04.608871 13090 heartbeater.cc:511] Master 127.1.151.189:44883 requested a full tablet report, sending...
I20260808 06:05:04.610069 12899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.585649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:54199 (local address 127.1.151.188:33465)
0808 06:05:04.585786 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:04.585791 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:04.586822 (+ 1031us) server_negotiation.cc:407] Connection header received
0808 06:05:04.588014 (+ 1192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:04.588017 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:04.588081 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:04.588148 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:04.589557 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.590430 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.592161 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.592504 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.592572 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.605167 (+ 12595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:04.605171 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:04.605174 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:04.605204 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:04.606725 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.607225 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.607229 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.607231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.607313 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.607686 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.607690 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.607692 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.607923 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:04.608000 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:04.609789 (+ 1789us) server_negotiation.cc:299] Negotiation successful
0808 06:05:04.609925 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 06:05:04.610124 12332 ts_manager.cc:194] Registered new tserver with Master: 1f756da4d21243bfbc204b1f2950800b (127.1.151.130:38501)
I20260808 06:05:04.610839 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.131:33501
--local_ip_for_outbound_sockets=127.1.151.131
--tserver_master_addrs=127.1.151.190:35293
--webserver_port=45859
--webserver_interface=127.1.151.131
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.131.keytab
--principal=oryx/127.1.151.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:45251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.151.190:35293,127.1.151.189:44883
--tserver_master_addrs=127.1.151.190:35293,127.1.151.189:44883,127.1.151.188:33465 with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 06:05:04.611234 13093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.586106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:33465 (local address 127.1.151.130:54199)
0808 06:05:04.586667 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:04.586719 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:04.587814 (+ 1095us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:04.588338 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:04.588344 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:04.588854 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:04.589410 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.589425 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.590596 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.590637 (+ 41us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.592022 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.592033 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.592734 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.592737 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.603087 (+ 10350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.603590 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:04.603605 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:04.604989 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:04.606848 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:04.606851 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:04.606853 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:04.607095 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:04.607425 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:04.607427 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:04.607430 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:04.607576 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:04.609513 (+ 1937us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:04.609516 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:04.609608 (+ 92us) client_negotiation.cc:769] Sending connection context
0808 06:05:04.610475 (+ 867us) client_negotiation.cc:240] Negotiation successful
0808 06:05:04.610641 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"mutex_wait_us":76,"spinlock_wait_cycles":59392,"thread_start_us":159,"threads_started":1}
I20260808 06:05:04.611807 13088 heartbeater.cc:347] Connected to a master server at 127.1.151.188:33465
I20260808 06:05:04.611891 13088 heartbeater.cc:464] Registering TS with master...
I20260808 06:05:04.612033 13088 heartbeater.cc:511] Master 127.1.151.188:33465 requested a full tablet report, sending...
I20260808 06:05:04.612736 11447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.585655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:36841 (local address 127.1.151.190:35293)
0808 06:05:04.585785 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:04.585789 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:04.586819 (+ 1030us) server_negotiation.cc:407] Connection header received
0808 06:05:04.588036 (+ 1217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:04.588040 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:04.588094 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:04.588158 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:04.590583 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.591222 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.592445 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.592737 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.592805 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.609316 (+ 16511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:04.609320 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:04.609323 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:04.609355 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:04.611017 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.611473 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.611477 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.611479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.611567 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.611874 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.611877 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.611879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.612088 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:04.612186 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:04.612507 (+ 321us) server_negotiation.cc:299] Negotiation successful
0808 06:05:04.612618 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 06:05:04.612787 13092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.586106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35293 (local address 127.1.151.130:36841)
0808 06:05:04.586666 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:04.586719 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:04.587812 (+ 1093us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:04.588340 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:04.588347 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:04.588860 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:04.589437 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.589448 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.591331 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.591335 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.592323 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.592332 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.592961 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.592964 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.593080 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.607057 (+ 13977us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:04.607071 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:04.609154 (+ 2083us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:04.611124 (+ 1970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:04.611128 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:04.611130 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:04.611375 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:04.611648 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:04.611650 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:04.611652 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:04.611744 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:04.612195 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:04.612198 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:04.612344 (+ 146us) client_negotiation.cc:769] Sending connection context
0808 06:05:04.612432 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 06:05:04.612561 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"mutex_wait_us":8,"spinlock_wait_cycles":53888,"thread_start_us":175,"threads_started":1}
I20260808 06:05:04.612985 12613 ts_manager.cc:194] Registered new tserver with Master: 1f756da4d21243bfbc204b1f2950800b (127.1.151.130:38501)
I20260808 06:05:04.613328 13089 heartbeater.cc:347] Connected to a master server at 127.1.151.190:35293
I20260808 06:05:04.613412 13089 heartbeater.cc:464] Registering TS with master...
I20260808 06:05:04.613555 13089 heartbeater.cc:511] Master 127.1.151.190:35293 requested a full tablet report, sending...
I20260808 06:05:04.614114 11384 ts_manager.cc:194] Re-registered known tserver with Master: 1f756da4d21243bfbc204b1f2950800b (127.1.151.130:38501)
I20260808 06:05:04.614689 11384 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.151.130@KRBTEST.COM'} at 127.1.151.130:36841
I20260808 06:05:04.704221 12266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.700547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:33465 (local address 127.0.0.1:41094)
0808 06:05:04.700704 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:04.700719 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:04.700815 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:04.701096 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:04.701098 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:04.701110 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:05:04.701205 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.701209 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.702259 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.702262 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.703019 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.703028 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.703910 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.703912 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.703981 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.704002 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 06:05:04.704044 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 06:05:04.704101 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 06:05:04.704309 12899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.700612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41094 (local address 127.1.151.188:33465)
0808 06:05:04.700792 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:04.700797 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:04.700817 (+ 20us) server_negotiation.cc:407] Connection header received
0808 06:05:04.700907 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:04.700911 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:04.700980 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:04.701082 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:05:04.701305 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.702085 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.703159 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.703803 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.703918 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.704072 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:04.704119 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 06:05:04.704183 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 06:05:04.704902 12626 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260808 06:05:04.705266 12683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88cc958608974ee987b790f318d04d52. Latest consensus state: current_term: 1 leader_uuid: "88cc958608974ee987b790f318d04d52" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: false } } peers { permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: NON_VOTER last_known_addr { host: "127.1.151.188" port: 33465 } attrs { promote: true } } }
I20260808 06:05:04.705413 12683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [sys.catalog]: This master's current role is: LEARNER
W20260808 06:05:04.723132 13095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:04.723347 13095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:04.723387 13095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:04.727205 13095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:04.727250 13095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:04.727304 13095 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.131
I20260808 06:05:04.732160 13095 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:45251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.131.keytab
--principal=oryx/127.1.151.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.131:33501
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.151.131
--webserver_port=45859
--tserver_master_addrs=127.1.151.190:35293,127.1.151.189:44883,127.1.151.188:33465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.131
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:05:04.733033 13095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:04.733935 13095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:04.743124 13108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:04.743134 13105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:04.743135 13106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:04.743249 13095 server_base.cc:1034] running on GCE node
I20260808 06:05:04.743778 13095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:04.744438 13095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:04.745590 13095 hybrid_clock.cc:648] HybridClock initialized: now 1786169104745568 us; error 43 us; skew 500 ppm
Aug 08 06:05:04 dist-test-slave-zj8q krb5kdc[11323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169104, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:04.748266 13095 init.cc:377] Logged in from keytab as oryx/127.1.151.131@KRBTEST.COM (short username oryx)
I20260808 06:05:04.749051 13095 webserver.cc:460] Webserver started at http://127.1.151.131:45859/ using document root <none> and password file <none>
I20260808 06:05:04.749552 13095 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:04.749620 13095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:04.753259 13095 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:05:04.755260 13117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:04.756376 13095 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 06:05:04.756474 13095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-2/wal
uuid: "543ae60e604c41868b2b7ab4a435673b"
format_stamp: "Formatted at 2026-08-08 06:05:03 on dist-test-slave-zj8q"
server_key: "3ed464d246348e3a0a1a531c16c4b84a"
server_key_iv: "c9aabb5229b211c594326ef2d5a6e467"
server_key_version: "encryptionkey@0"
I20260808 06:05:04.757033 13095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:04.770905 13095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:04.773945 13095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:05:04.774096 13095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:04.774688 13095 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:05:04.775547 13095 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:05:04.775595 13095 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:04.775657 13095 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:05:04.775682 13095 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:04.784247 13095 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.131:33501
I20260808 06:05:04.784271 13291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.131:33501 every 8 connection(s)
I20260808 06:05:04.785187 13095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
I20260808 06:05:04.788012 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 13095
Aug 08 06:05:04 dist-test-slave-zj8q krb5kdc[11323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169104, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.188@KRBTEST.COM
Aug 08 06:05:04 dist-test-slave-zj8q krb5kdc[11323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169104, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM
Aug 08 06:05:04 dist-test-slave-zj8q krb5kdc[11323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169104, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.189@KRBTEST.COM
I20260808 06:05:04.802294 12899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.788623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:56313 (local address 127.1.151.188:33465)
0808 06:05:04.788759 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:04.788764 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:04.789552 (+ 788us) server_negotiation.cc:407] Connection header received
0808 06:05:04.790918 (+ 1366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:04.790922 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:04.790972 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:04.791046 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:04.792494 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.793272 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.794503 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.794828 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.794903 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.798192 (+ 3289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:04.798195 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:04.798198 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:04.798224 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:04.799807 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.800414 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.800417 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.800419 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.800514 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.800929 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.800933 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.800935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.801169 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:04.801242 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:04.801963 (+ 721us) server_negotiation.cc:299] Negotiation successful
0808 06:05:04.802087 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54,"spinlock_wait_cycles":5376}
I20260808 06:05:04.803877 13298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.787390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35293 (local address 127.1.151.131:34667)
0808 06:05:04.789494 (+ 2104us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:04.789508 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:04.789633 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:04.789988 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:04.789991 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:04.790236 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:04.791019 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.791027 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.794315 (+ 3288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.794318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.795093 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.795103 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.795763 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.795767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.795874 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.796273 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:04.796287 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:04.798376 (+ 2089us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:04.800962 (+ 2586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:04.800965 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:04.800967 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:04.801216 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:04.801540 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:04.801542 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:04.801544 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:04.801634 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:04.802130 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:04.802133 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:04.802253 (+ 120us) client_negotiation.cc:769] Sending connection context
0808 06:05:04.802365 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 06:05:04.802772 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1778,"mutex_wait_us":601,"spinlock_wait_cycles":44544,"thread_start_us":157,"threads_started":1}
I20260808 06:05:04.804569 11447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.786933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:34667 (local address 127.1.151.190:35293)
0808 06:05:04.787198 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:04.787202 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:04.789567 (+ 2365us) server_negotiation.cc:407] Connection header received
0808 06:05:04.789739 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:04.789742 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:04.789793 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:04.789843 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:04.791166 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.791908 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.795226 (+ 3318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.795514 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.795587 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.798785 (+ 3198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:04.798789 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:04.798791 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:04.798818 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:04.800829 (+ 2011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.801334 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.801338 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.801340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.801426 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.801761 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.801765 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.801767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.801983 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:04.802067 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:04.804307 (+ 2240us) server_negotiation.cc:299] Negotiation successful
0808 06:05:04.804417 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192}
I20260808 06:05:04.804711 13312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.790418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59594 (local address 127.1.151.189:44883)
0808 06:05:04.790650 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:04.790655 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:04.791442 (+ 787us) server_negotiation.cc:407] Connection header received
0808 06:05:04.791487 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:04.791490 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:04.791538 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:04.791637 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:04.792496 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.793285 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.794464 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.794804 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.794891 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.796626 (+ 1735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:04.796629 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:04.796632 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:04.796658 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:04.801379 (+ 4721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.803067 (+ 1688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.803071 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.803073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.803165 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.803485 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.803489 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.803490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.803738 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:04.803864 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:04.804408 (+ 544us) server_negotiation.cc:299] Negotiation successful
0808 06:05:04.804535 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":83,"threads_started":1}
I20260808 06:05:04.804831 13308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.788415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:33465 (local address 127.1.151.131:56313)
0808 06:05:04.789463 (+ 1048us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:04.789476 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:04.789621 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:04.792049 (+ 2428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:04.792052 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:04.792233 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:04.792351 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.792359 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.793410 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.793412 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.794379 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.794387 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.795057 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.795061 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.795256 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.795832 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:04.795854 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:04.797978 (+ 2124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:04.799933 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:04.799940 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:04.799956 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:04.800302 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:04.800627 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:04.800630 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:04.800632 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:04.800820 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:04.801328 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:04.801336 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:04.801664 (+ 328us) client_negotiation.cc:769] Sending connection context
0808 06:05:04.802494 (+ 830us) client_negotiation.cc:240] Negotiation successful
0808 06:05:04.804664 (+ 2170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":982,"mutex_wait_us":2781,"thread_start_us":83,"threads_started":1}
I20260808 06:05:04.805850 13292 heartbeater.cc:347] Connected to a master server at 127.1.151.188:33465
I20260808 06:05:04.806097 13292 heartbeater.cc:464] Registering TS with master...
I20260808 06:05:04.806114 13293 heartbeater.cc:347] Connected to a master server at 127.1.151.190:35293
I20260808 06:05:04.806258 13293 heartbeater.cc:464] Registering TS with master...
I20260808 06:05:04.806622 13316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.790956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41100 (local address 127.1.151.188:33465)
0808 06:05:04.795397 (+ 4441us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:04.795412 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:04.797000 (+ 1588us) server_negotiation.cc:407] Connection header received
0808 06:05:04.797038 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:04.797040 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:04.797080 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:04.797155 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:04.798131 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.798888 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.799987 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.800312 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.800396 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.802498 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:04.802502 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:04.802504 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:04.802526 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:04.804494 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.805224 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.805227 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.805229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.805316 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.805636 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.805640 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.805641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.805839 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:04.805966 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:04.806345 (+ 379us) server_negotiation.cc:299] Negotiation successful
0808 06:05:04.806462 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4373,"thread_start_us":91,"threads_started":1}
I20260808 06:05:04.806756 13293 heartbeater.cc:511] Master 127.1.151.190:35293 requested a full tablet report, sending...
I20260808 06:05:04.807358 12524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.786933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:46327 (local address 127.1.151.189:44883)
0808 06:05:04.787099 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:04.787104 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:04.788058 (+ 954us) server_negotiation.cc:407] Connection header received
0808 06:05:04.789084 (+ 1026us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:04.789087 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:04.789147 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:04.789232 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:04.791153 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.791937 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.793549 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.793887 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.793957 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.799822 (+ 5865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:04.799826 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:04.799829 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:04.799856 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:04.802336 (+ 2480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.803839 (+ 1503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.803842 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.803844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.803934 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.805684 (+ 1750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.805687 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.805689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.805892 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:04.805965 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:04.807109 (+ 1144us) server_negotiation.cc:299] Negotiation successful
0808 06:05:04.807222 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 06:05:04.806753 13292 heartbeater.cc:511] Master 127.1.151.188:33465 requested a full tablet report, sending...
I20260808 06:05:04.808003 11384 ts_manager.cc:194] Re-registered known tserver with Master: 543ae60e604c41868b2b7ab4a435673b (127.1.151.131:33501)
I20260808 06:05:04.808146 13296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.787390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.189:44883 (local address 127.1.151.131:46327)
0808 06:05:04.787914 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:04.787964 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:04.788860 (+ 896us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:04.789538 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:04.789546 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:04.790079 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:04.790800 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.790820 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.792108 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.792111 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.793419 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.793430 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.794146 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.794170 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.795560 (+ 1390us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.796665 (+ 1105us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:04.796681 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:04.798960 (+ 2279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:04.802765 (+ 3805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:04.803492 (+ 727us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:04.803495 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:04.803721 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:04.804646 (+ 925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:04.804649 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:04.804651 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:04.804740 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:04.806040 (+ 1300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:04.806042 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:04.806919 (+ 877us) client_negotiation.cc:769] Sending connection context
0808 06:05:04.807562 (+ 643us) client_negotiation.cc:240] Negotiation successful
0808 06:05:04.807710 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":5,"thread_start_us":187,"threads_started":1}
I20260808 06:05:04.808458 13310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.789966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57846 (local address 127.1.151.190:35293)
0808 06:05:04.790241 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:04.790245 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:04.790637 (+ 392us) server_negotiation.cc:407] Connection header received
0808 06:05:04.790823 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:04.790826 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:04.790875 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:04.790956 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:04.792033 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.792840 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.793990 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.794381 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.794460 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.800210 (+ 5750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:04.800214 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:04.800217 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:04.800246 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:04.803412 (+ 3166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.806676 (+ 3264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.806680 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.806682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.806774 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:04.807209 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:04.807212 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:04.807214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:04.807421 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:04.807534 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:04.808167 (+ 633us) server_negotiation.cc:299] Negotiation successful
0808 06:05:04.808289 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":82,"threads_started":1}
I20260808 06:05:04.808698 11384 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.151.131@KRBTEST.COM'} at 127.1.151.131:34667
I20260808 06:05:04.808736 12613 ts_manager.cc:194] Registered new tserver with Master: 543ae60e604c41868b2b7ab4a435673b (127.1.151.131:33501)
I20260808 06:05:04.808912 13294 heartbeater.cc:347] Connected to a master server at 127.1.151.189:44883
I20260808 06:05:04.808987 13294 heartbeater.cc:464] Registering TS with master...
I20260808 06:05:04.809122 13294 heartbeater.cc:511] Master 127.1.151.189:44883 requested a full tablet report, sending...
I20260808 06:05:04.809737 12332 ts_manager.cc:194] Registered new tserver with Master: 543ae60e604c41868b2b7ab4a435673b (127.1.151.131:33501)
I20260808 06:05:04.811519 11384 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57846:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 06:05:04.814352 11384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 06:05:04.816740 12576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "88cc958608974ee987b790f318d04d52" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: false } } peers { permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: NON_VOTER last_known_addr { host: "127.1.151.188" port: 33465 } attrs { promote: true } } }
I20260808 06:05:04.816861 12576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [sys.catalog]: This master's current role is: LEADER
I20260808 06:05:04.819866 12576 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52: attempting to promote NON_VOTER 4275957a7be24a9cb4d20a46934a9d1f to VOTER
I20260808 06:05:04.820255 12576 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: false } } peers { permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 33465 } attrs { promote: false } }
I20260808 06:05:04.821889 12350 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [term 1 FOLLOWER]: Refusing update from remote peer 88cc958608974ee987b790f318d04d52: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 06:05:04.821889 12626 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [term 1 LEARNER]: Refusing update from remote peer 88cc958608974ee987b790f318d04d52: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 06:05:04.822588 13331 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 06:05:04.822767 13318 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 33465 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 06:05:04.824179 13318 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 4275957a7be24a9cb4d20a46934a9d1f (127.1.151.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: false } } peers { permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 33465 } attrs { promote: false } } }
I20260808 06:05:04.824658 12350 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 4275957a7be24a9cb4d20a46934a9d1f (127.1.151.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: false } } peers { permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 33465 } attrs { promote: false } } }
I20260808 06:05:04.825244 12626 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 4275957a7be24a9cb4d20a46934a9d1f (127.1.151.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: false } } peers { permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 33465 } attrs { promote: false } } }
I20260808 06:05:04.825937 13318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "88cc958608974ee987b790f318d04d52" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: false } } peers { permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 33465 } attrs { promote: false } } }
I20260808 06:05:04.826084 13318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [sys.catalog]: This master's current role is: LEADER
I20260808 06:05:04.826359 13318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "88cc958608974ee987b790f318d04d52" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: false } } peers { permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 33465 } attrs { promote: false } } }
I20260808 06:05:04.826494 13318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [sys.catalog]: This master's current role is: LEADER
I20260808 06:05:04.826344 13320 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "88cc958608974ee987b790f318d04d52" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: false } } peers { permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 33465 } attrs { promote: false } } }
I20260808 06:05:04.826586 13320 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52 [sys.catalog]: This master's current role is: LEADER
I20260808 06:05:04.828833 12683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "88cc958608974ee987b790f318d04d52" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: false } } peers { permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 33465 } attrs { promote: false } } }
I20260808 06:05:04.828950 12683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 06:05:04.831753 13332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "88cc958608974ee987b790f318d04d52" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: false } } peers { permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 33465 } attrs { promote: false } } }
I20260808 06:05:04.831874 13332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 06:05:04.834486 13336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.828871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:33501 (local address 127.0.0.1:53112)
0808 06:05:04.829387 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:04.829401 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:04.829488 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:04.830377 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:04.830380 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:04.830396 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:05:04.830496 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.830502 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.831559 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.831563 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.832914 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.832925 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.834064 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.834068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.834201 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.834225 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 06:05:04.834286 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 06:05:04.834354 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":92,"threads_started":1}
I20260808 06:05:04.834744 13338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.829552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53112 (local address 127.1.151.131:33501)
0808 06:05:04.829847 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:04.829858 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:04.829882 (+ 24us) server_negotiation.cc:407] Connection header received
0808 06:05:04.829960 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:04.829965 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:04.830215 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:04.830412 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:05:04.830584 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.831445 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.833097 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.833956 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.834052 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.834444 (+ 392us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:04.834495 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 06:05:04.834589 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":96,"threads_started":1}
I20260808 06:05:04.835700 12266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.828644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:38287 (local address 127.0.0.1:46298)
0808 06:05:04.828818 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:04.828833 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:04.828911 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:04.831529 (+ 2618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:04.831532 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:04.831548 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:05:04.831646 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.831653 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.833077 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.833081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.834237 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.834249 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.835309 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.835313 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.835397 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.835419 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 06:05:04.835501 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 06:05:04.835585 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101}
I20260808 06:05:04.835772 13333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.828804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46298 (local address 127.1.151.129:38287)
0808 06:05:04.831180 (+ 2376us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:04.831189 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:04.831209 (+ 20us) server_negotiation.cc:407] Connection header received
0808 06:05:04.831271 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:04.831275 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:04.831405 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:04.831531 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:05:04.831771 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.832971 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.834376 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.835191 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.835298 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.835428 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:04.835528 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 06:05:04.835620 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2272,"thread_start_us":94,"threads_started":1}
I20260808 06:05:04.836239 13335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.829173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:38501 (local address 127.0.0.1:36164)
0808 06:05:04.830112 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:04.830125 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:04.830932 (+ 807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:04.832467 (+ 1535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:04.832470 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:04.832487 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:05:04.832585 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.832591 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.833788 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.833791 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.834898 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.834909 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.835937 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.835941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.836029 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.836048 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 06:05:04.836086 (+ 38us) client_negotiation.cc:240] Negotiation successful
0808 06:05:04.836133 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":758,"thread_start_us":87,"threads_started":1}
I20260808 06:05:04.836728 13337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.829336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36164 (local address 127.1.151.130:38501)
0808 06:05:04.832119 (+ 2783us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:04.832128 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:04.832154 (+ 26us) server_negotiation.cc:407] Connection header received
0808 06:05:04.832215 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:04.832218 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:04.832343 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:04.832467 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:05:04.832679 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.833661 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.835047 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.835798 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.836401 (+ 603us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.836518 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:04.836556 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 06:05:04.836618 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2696,"thread_start_us":186,"threads_started":1}
I20260808 06:05:04.837890 13187 tablet_service.cc:1467] Processing CreateTablet for tablet 58370956386b406eaf3f8db75e13cdff (DEFAULT_TABLE table=test-table [id=7eff9fcc80a24977bd4939fa87d2ccc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:05:04.838990 13187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58370956386b406eaf3f8db75e13cdff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:05:04.838826 12995 tablet_service.cc:1467] Processing CreateTablet for tablet 58370956386b406eaf3f8db75e13cdff (DEFAULT_TABLE table=test-table [id=7eff9fcc80a24977bd4939fa87d2ccc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:05:04.839589 12995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58370956386b406eaf3f8db75e13cdff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:05:04.844355 12794 tablet_service.cc:1467] Processing CreateTablet for tablet 58370956386b406eaf3f8db75e13cdff (DEFAULT_TABLE table=test-table [id=7eff9fcc80a24977bd4939fa87d2ccc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:05:04.845386 12794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58370956386b406eaf3f8db75e13cdff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:05:04.846650 13340 tablet_bootstrap.cc:492] T 58370956386b406eaf3f8db75e13cdff P 1f756da4d21243bfbc204b1f2950800b: Bootstrap starting.
I20260808 06:05:04.847854 13341 tablet_bootstrap.cc:492] T 58370956386b406eaf3f8db75e13cdff P 543ae60e604c41868b2b7ab4a435673b: Bootstrap starting.
I20260808 06:05:04.849123 13340 tablet_bootstrap.cc:654] T 58370956386b406eaf3f8db75e13cdff P 1f756da4d21243bfbc204b1f2950800b: Neither blocks nor log segments found. Creating new log.
I20260808 06:05:04.850217 13341 tablet_bootstrap.cc:654] T 58370956386b406eaf3f8db75e13cdff P 543ae60e604c41868b2b7ab4a435673b: Neither blocks nor log segments found. Creating new log.
I20260808 06:05:04.850252 13340 log.cc:824] T 58370956386b406eaf3f8db75e13cdff P 1f756da4d21243bfbc204b1f2950800b: Log is configured to *not* fsync() on all Append() calls
I20260808 06:05:04.851089 13341 log.cc:824] T 58370956386b406eaf3f8db75e13cdff P 543ae60e604c41868b2b7ab4a435673b: Log is configured to *not* fsync() on all Append() calls
I20260808 06:05:04.852916 13341 tablet_bootstrap.cc:492] T 58370956386b406eaf3f8db75e13cdff P 543ae60e604c41868b2b7ab4a435673b: No bootstrap required, opened a new log
I20260808 06:05:04.853107 13341 ts_tablet_manager.cc:1397] T 58370956386b406eaf3f8db75e13cdff P 543ae60e604c41868b2b7ab4a435673b: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260808 06:05:04.854823 13340 tablet_bootstrap.cc:492] T 58370956386b406eaf3f8db75e13cdff P 1f756da4d21243bfbc204b1f2950800b: No bootstrap required, opened a new log
I20260808 06:05:04.855019 13340 ts_tablet_manager.cc:1397] T 58370956386b406eaf3f8db75e13cdff P 1f756da4d21243bfbc204b1f2950800b: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.001s
I20260808 06:05:04.857530 13345 tablet_bootstrap.cc:492] T 58370956386b406eaf3f8db75e13cdff P a67dd2b3f83143dd958e5cb489a649c2: Bootstrap starting.
I20260808 06:05:04.858062 13341 raft_consensus.cc:348] T 58370956386b406eaf3f8db75e13cdff P 543ae60e604c41868b2b7ab4a435673b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "543ae60e604c41868b2b7ab4a435673b" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33501 } } peers { permanent_uuid: "a67dd2b3f83143dd958e5cb489a649c2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 38287 } } peers { permanent_uuid: "1f756da4d21243bfbc204b1f2950800b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 38501 } }
I20260808 06:05:04.858558 13341 raft_consensus.cc:374] T 58370956386b406eaf3f8db75e13cdff P 543ae60e604c41868b2b7ab4a435673b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:05:04.858669 13341 raft_consensus.cc:729] T 58370956386b406eaf3f8db75e13cdff P 543ae60e604c41868b2b7ab4a435673b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 543ae60e604c41868b2b7ab4a435673b, State: Initialized, Role: FOLLOWER
I20260808 06:05:04.859302 13341 consensus_queue.cc:260] T 58370956386b406eaf3f8db75e13cdff P 543ae60e604c41868b2b7ab4a435673b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "543ae60e604c41868b2b7ab4a435673b" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33501 } } peers { permanent_uuid: "a67dd2b3f83143dd958e5cb489a649c2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 38287 } } peers { permanent_uuid: "1f756da4d21243bfbc204b1f2950800b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 38501 } }
I20260808 06:05:04.860191 13345 tablet_bootstrap.cc:654] T 58370956386b406eaf3f8db75e13cdff P a67dd2b3f83143dd958e5cb489a649c2: Neither blocks nor log segments found. Creating new log.
I20260808 06:05:04.860407 13341 ts_tablet_manager.cc:1428] T 58370956386b406eaf3f8db75e13cdff P 543ae60e604c41868b2b7ab4a435673b: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260808 06:05:04.860774 13293 heartbeater.cc:503] Master 127.1.151.190:35293 was elected leader, sending a full tablet report...
I20260808 06:05:04.861296 13345 log.cc:824] T 58370956386b406eaf3f8db75e13cdff P a67dd2b3f83143dd958e5cb489a649c2: Log is configured to *not* fsync() on all Append() calls
I20260808 06:05:04.861151 13340 raft_consensus.cc:348] T 58370956386b406eaf3f8db75e13cdff P 1f756da4d21243bfbc204b1f2950800b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "543ae60e604c41868b2b7ab4a435673b" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33501 } } peers { permanent_uuid: "a67dd2b3f83143dd958e5cb489a649c2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 38287 } } peers { permanent_uuid: "1f756da4d21243bfbc204b1f2950800b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 38501 } }
I20260808 06:05:04.861474 13340 raft_consensus.cc:374] T 58370956386b406eaf3f8db75e13cdff P 1f756da4d21243bfbc204b1f2950800b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:05:04.861523 13340 raft_consensus.cc:729] T 58370956386b406eaf3f8db75e13cdff P 1f756da4d21243bfbc204b1f2950800b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f756da4d21243bfbc204b1f2950800b, State: Initialized, Role: FOLLOWER
I20260808 06:05:04.863535 13345 tablet_bootstrap.cc:492] T 58370956386b406eaf3f8db75e13cdff P a67dd2b3f83143dd958e5cb489a649c2: No bootstrap required, opened a new log
I20260808 06:05:04.863618 13340 consensus_queue.cc:260] T 58370956386b406eaf3f8db75e13cdff P 1f756da4d21243bfbc204b1f2950800b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "543ae60e604c41868b2b7ab4a435673b" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33501 } } peers { permanent_uuid: "a67dd2b3f83143dd958e5cb489a649c2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 38287 } } peers { permanent_uuid: "1f756da4d21243bfbc204b1f2950800b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 38501 } }
I20260808 06:05:04.863734 13345 ts_tablet_manager.cc:1397] T 58370956386b406eaf3f8db75e13cdff P a67dd2b3f83143dd958e5cb489a649c2: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 06:05:04.864408 13089 heartbeater.cc:503] Master 127.1.151.190:35293 was elected leader, sending a full tablet report...
I20260808 06:05:04.864677 13340 ts_tablet_manager.cc:1428] T 58370956386b406eaf3f8db75e13cdff P 1f756da4d21243bfbc204b1f2950800b: Time spent starting tablet: real 0.010s user 0.000s sys 0.008s
I20260808 06:05:04.870142 13345 raft_consensus.cc:348] T 58370956386b406eaf3f8db75e13cdff P a67dd2b3f83143dd958e5cb489a649c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "543ae60e604c41868b2b7ab4a435673b" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33501 } } peers { permanent_uuid: "a67dd2b3f83143dd958e5cb489a649c2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 38287 } } peers { permanent_uuid: "1f756da4d21243bfbc204b1f2950800b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 38501 } }
I20260808 06:05:04.870398 13345 raft_consensus.cc:374] T 58370956386b406eaf3f8db75e13cdff P a67dd2b3f83143dd958e5cb489a649c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:05:04.870450 13345 raft_consensus.cc:729] T 58370956386b406eaf3f8db75e13cdff P a67dd2b3f83143dd958e5cb489a649c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a67dd2b3f83143dd958e5cb489a649c2, State: Initialized, Role: FOLLOWER
I20260808 06:05:04.870831 13345 consensus_queue.cc:260] T 58370956386b406eaf3f8db75e13cdff P a67dd2b3f83143dd958e5cb489a649c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "543ae60e604c41868b2b7ab4a435673b" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33501 } } peers { permanent_uuid: "a67dd2b3f83143dd958e5cb489a649c2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 38287 } } peers { permanent_uuid: "1f756da4d21243bfbc204b1f2950800b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 38501 } }
I20260808 06:05:04.871527 12892 heartbeater.cc:503] Master 127.1.151.190:35293 was elected leader, sending a full tablet report...
I20260808 06:05:04.871668 13345 ts_tablet_manager.cc:1428] T 58370956386b406eaf3f8db75e13cdff P a67dd2b3f83143dd958e5cb489a649c2: Time spent starting tablet: real 0.008s user 0.002s sys 0.004s
W20260808 06:05:04.893496 12895 tablet.cc:2369] T 58370956386b406eaf3f8db75e13cdff P a67dd2b3f83143dd958e5cb489a649c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 06:05:04.926133 13349 raft_consensus.cc:482] T 58370956386b406eaf3f8db75e13cdff P 1f756da4d21243bfbc204b1f2950800b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:05:04.926388 13349 raft_consensus.cc:504] T 58370956386b406eaf3f8db75e13cdff P 1f756da4d21243bfbc204b1f2950800b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "543ae60e604c41868b2b7ab4a435673b" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33501 } } peers { permanent_uuid: "a67dd2b3f83143dd958e5cb489a649c2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 38287 } } peers { permanent_uuid: "1f756da4d21243bfbc204b1f2950800b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 38501 } }
I20260808 06:05:04.927525 13349 leader_election.cc:288] T 58370956386b406eaf3f8db75e13cdff P 1f756da4d21243bfbc204b1f2950800b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 543ae60e604c41868b2b7ab4a435673b (127.1.151.131:33501), a67dd2b3f83143dd958e5cb489a649c2 (127.1.151.129:38287)
I20260808 06:05:04.931159 13338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.927876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:59825 (local address 127.1.151.131:33501)
0808 06:05:04.928017 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:04.928022 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:04.928091 (+ 69us) server_negotiation.cc:407] Connection header received
0808 06:05:04.928219 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:04.928222 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:04.928262 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:04.928323 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:05:04.928562 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.929082 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.930182 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.930572 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.930683 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.930737 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:04.930952 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 06:05:04.931024 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 06:05:04.931176 13333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.927875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:54753 (local address 127.1.151.129:38287)
0808 06:05:04.927993 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:04.927998 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:04.928011 (+ 13us) server_negotiation.cc:407] Connection header received
0808 06:05:04.928066 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:04.928069 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:04.928111 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:04.928179 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:05:04.928577 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.929072 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.930194 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.930578 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.930685 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.930737 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:04.930986 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 06:05:04.931061 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 06:05:04.931190 13093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.927741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:33501 (local address 127.1.151.130:59825)
0808 06:05:04.928013 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:04.928027 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:04.928121 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:04.928340 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:04.928343 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:04.928359 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:05:04.928465 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.928470 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.929186 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.929190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.930026 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.930036 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.930689 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.930692 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.930839 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.930859 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 06:05:04.930924 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 06:05:04.931002 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":185}
I20260808 06:05:04.931274 13092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.927741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:38287 (local address 127.1.151.130:54753)
0808 06:05:04.927877 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:04.927893 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:04.927986 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:04.928207 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:04.928210 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:04.928226 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:05:04.928447 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.928455 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.929180 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.929184 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.930044 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:04.930054 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.930700 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.930703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:04.930869 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.930887 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 06:05:04.930955 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 06:05:04.931025 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":39}
I20260808 06:05:04.931911 12821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58370956386b406eaf3f8db75e13cdff" candidate_uuid: "1f756da4d21243bfbc204b1f2950800b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a67dd2b3f83143dd958e5cb489a649c2" is_pre_election: true
I20260808 06:05:04.931937 13218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58370956386b406eaf3f8db75e13cdff" candidate_uuid: "1f756da4d21243bfbc204b1f2950800b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "543ae60e604c41868b2b7ab4a435673b" is_pre_election: true
I20260808 06:05:04.932235 13218 raft_consensus.cc:2454] T 58370956386b406eaf3f8db75e13cdff P 543ae60e604c41868b2b7ab4a435673b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f756da4d21243bfbc204b1f2950800b in term 0.
I20260808 06:05:04.932235 12821 raft_consensus.cc:2454] T 58370956386b406eaf3f8db75e13cdff P a67dd2b3f83143dd958e5cb489a649c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f756da4d21243bfbc204b1f2950800b in term 0.
I20260808 06:05:04.932721 12931 leader_election.cc:302] T 58370956386b406eaf3f8db75e13cdff P 1f756da4d21243bfbc204b1f2950800b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f756da4d21243bfbc204b1f2950800b, a67dd2b3f83143dd958e5cb489a649c2; no voters:
I20260808 06:05:04.932976 13349 raft_consensus.cc:2781] T 58370956386b406eaf3f8db75e13cdff P 1f756da4d21243bfbc204b1f2950800b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 06:05:04.933060 13349 raft_consensus.cc:482] T 58370956386b406eaf3f8db75e13cdff P 1f756da4d21243bfbc204b1f2950800b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 06:05:04.933111 13349 raft_consensus.cc:3037] T 58370956386b406eaf3f8db75e13cdff P 1f756da4d21243bfbc204b1f2950800b [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:05:04.934557 13349 raft_consensus.cc:504] T 58370956386b406eaf3f8db75e13cdff P 1f756da4d21243bfbc204b1f2950800b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "543ae60e604c41868b2b7ab4a435673b" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33501 } } peers { permanent_uuid: "a67dd2b3f83143dd958e5cb489a649c2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 38287 } } peers { permanent_uuid: "1f756da4d21243bfbc204b1f2950800b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 38501 } }
I20260808 06:05:04.934871 13349 leader_election.cc:288] T 58370956386b406eaf3f8db75e13cdff P 1f756da4d21243bfbc204b1f2950800b [CANDIDATE]: Term 1 election: Requested vote from peers 543ae60e604c41868b2b7ab4a435673b (127.1.151.131:33501), a67dd2b3f83143dd958e5cb489a649c2 (127.1.151.129:38287)
I20260808 06:05:04.935206 13218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58370956386b406eaf3f8db75e13cdff" candidate_uuid: "1f756da4d21243bfbc204b1f2950800b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "543ae60e604c41868b2b7ab4a435673b"
I20260808 06:05:04.935213 12821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58370956386b406eaf3f8db75e13cdff" candidate_uuid: "1f756da4d21243bfbc204b1f2950800b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a67dd2b3f83143dd958e5cb489a649c2"
I20260808 06:05:04.935324 12821 raft_consensus.cc:3037] T 58370956386b406eaf3f8db75e13cdff P a67dd2b3f83143dd958e5cb489a649c2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:05:04.935336 13218 raft_consensus.cc:3037] T 58370956386b406eaf3f8db75e13cdff P 543ae60e604c41868b2b7ab4a435673b [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:05:04.936391 12821 raft_consensus.cc:2454] T 58370956386b406eaf3f8db75e13cdff P a67dd2b3f83143dd958e5cb489a649c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f756da4d21243bfbc204b1f2950800b in term 1.
I20260808 06:05:04.936668 12931 leader_election.cc:302] T 58370956386b406eaf3f8db75e13cdff P 1f756da4d21243bfbc204b1f2950800b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f756da4d21243bfbc204b1f2950800b, a67dd2b3f83143dd958e5cb489a649c2; no voters:
I20260808 06:05:04.936873 13349 raft_consensus.cc:2781] T 58370956386b406eaf3f8db75e13cdff P 1f756da4d21243bfbc204b1f2950800b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:05:04.936924 13218 raft_consensus.cc:2454] T 58370956386b406eaf3f8db75e13cdff P 543ae60e604c41868b2b7ab4a435673b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f756da4d21243bfbc204b1f2950800b in term 1.
I20260808 06:05:04.937153 13349 raft_consensus.cc:686] T 58370956386b406eaf3f8db75e13cdff P 1f756da4d21243bfbc204b1f2950800b [term 1 LEADER]: Becoming Leader. State: Replica: 1f756da4d21243bfbc204b1f2950800b, State: Running, Role: LEADER
I20260808 06:05:04.937731 13349 consensus_queue.cc:237] T 58370956386b406eaf3f8db75e13cdff P 1f756da4d21243bfbc204b1f2950800b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "543ae60e604c41868b2b7ab4a435673b" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33501 } } peers { permanent_uuid: "a67dd2b3f83143dd958e5cb489a649c2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 38287 } } peers { permanent_uuid: "1f756da4d21243bfbc204b1f2950800b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 38501 } }
I20260808 06:05:04.941390 11383 catalog_manager.cc:5495] T 58370956386b406eaf3f8db75e13cdff P 1f756da4d21243bfbc204b1f2950800b reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f756da4d21243bfbc204b1f2950800b (127.1.151.130). New cstate: current_term: 1 leader_uuid: "1f756da4d21243bfbc204b1f2950800b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "543ae60e604c41868b2b7ab4a435673b" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a67dd2b3f83143dd958e5cb489a649c2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 38287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f756da4d21243bfbc204b1f2950800b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 38501 } health_report { overall_health: HEALTHY } } }
I20260808 06:05:04.956462 13337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.953040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36168 (local address 127.1.151.130:38501)
0808 06:05:04.953166 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:04.953169 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:04.953182 (+ 13us) server_negotiation.cc:407] Connection header received
0808 06:05:04.953252 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:04.953254 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:04.953304 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:04.953390 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:05:04.953715 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.954311 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.955114 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:04.955318 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:04.955390 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:04.955742 (+ 352us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:05:04.956160 (+ 418us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:05:04.956253 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:04.956299 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 06:05:04.956351 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 06:05:04.962738 13218 raft_consensus.cc:1261] T 58370956386b406eaf3f8db75e13cdff P 543ae60e604c41868b2b7ab4a435673b [term 1 FOLLOWER]: Refusing update from remote peer 1f756da4d21243bfbc204b1f2950800b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:05:04.962781 12821 raft_consensus.cc:1261] T 58370956386b406eaf3f8db75e13cdff P a67dd2b3f83143dd958e5cb489a649c2 [term 1 FOLLOWER]: Refusing update from remote peer 1f756da4d21243bfbc204b1f2950800b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:05:04.963447 13351 consensus_queue.cc:1035] T 58370956386b406eaf3f8db75e13cdff P 1f756da4d21243bfbc204b1f2950800b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a67dd2b3f83143dd958e5cb489a649c2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 38287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:05:04.963618 13349 consensus_queue.cc:1035] T 58370956386b406eaf3f8db75e13cdff P 1f756da4d21243bfbc204b1f2950800b [LEADER]: Connected to new peer: Peer: permanent_uuid: "543ae60e604c41868b2b7ab4a435673b" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:05:04.972704 13360 mvcc.cc:205] Tried to move back new op lower bound from 7316148653920366592 to 7316148653827616768. Current Snapshot: MvccSnapshot[applied={T|T < 7316148653920366592}]
I20260808 06:05:04.980814 11382 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:05:04.980950 11382 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:05:04.984647 11382 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 88cc958608974ee987b790f318d04d52: Sending DeleteTablet for 3 replicas of tablet 58370956386b406eaf3f8db75e13cdff
I20260808 06:05:04.985729 12995 tablet_service.cc:1514] Processing DeleteTablet for tablet 58370956386b406eaf3f8db75e13cdff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:05:04 UTC) from {username='oryx', principal='oryx/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:36164
I20260808 06:05:04.986255 12794 tablet_service.cc:1514] Processing DeleteTablet for tablet 58370956386b406eaf3f8db75e13cdff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:05:04 UTC) from {username='oryx', principal='oryx/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:46298
I20260808 06:05:04.987218 13375 tablet_replica.cc:330] T 58370956386b406eaf3f8db75e13cdff P a67dd2b3f83143dd958e5cb489a649c2: stopping tablet replica
I20260808 06:05:04.987558 13375 raft_consensus.cc:2229] T 58370956386b406eaf3f8db75e13cdff P a67dd2b3f83143dd958e5cb489a649c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:05:04.987947 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 11344
I20260808 06:05:04.987964 13375 raft_consensus.cc:2258] T 58370956386b406eaf3f8db75e13cdff P a67dd2b3f83143dd958e5cb489a649c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:05:04.987942 13187 tablet_service.cc:1514] Processing DeleteTablet for tablet 58370956386b406eaf3f8db75e13cdff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:05:04 UTC) from {username='oryx', principal='oryx/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:53112
I20260808 06:05:04.994055 13377 tablet_replica.cc:330] T 58370956386b406eaf3f8db75e13cdff P 543ae60e604c41868b2b7ab4a435673b: stopping tablet replica
I20260808 06:05:04.994441 13377 raft_consensus.cc:2229] T 58370956386b406eaf3f8db75e13cdff P 543ae60e604c41868b2b7ab4a435673b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:05:04.994774 13377 raft_consensus.cc:2258] T 58370956386b406eaf3f8db75e13cdff P 543ae60e604c41868b2b7ab4a435673b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:05:04.994961 13375 ts_tablet_manager.cc:1905] T 58370956386b406eaf3f8db75e13cdff P a67dd2b3f83143dd958e5cb489a649c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:05:04.997565 13377 ts_tablet_manager.cc:1905] T 58370956386b406eaf3f8db75e13cdff P 543ae60e604c41868b2b7ab4a435673b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:05:04.998009 13375 ts_tablet_manager.cc:1918] T 58370956386b406eaf3f8db75e13cdff P a67dd2b3f83143dd958e5cb489a649c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:05:04.998202 13375 log.cc:1196] T 58370956386b406eaf3f8db75e13cdff P a67dd2b3f83143dd958e5cb489a649c2: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-0/wal/wals/58370956386b406eaf3f8db75e13cdff
I20260808 06:05:04.998620 13375 ts_tablet_manager.cc:1939] T 58370956386b406eaf3f8db75e13cdff P a67dd2b3f83143dd958e5cb489a649c2: Deleting consensus metadata
I20260808 06:05:05.000308 13377 ts_tablet_manager.cc:1918] T 58370956386b406eaf3f8db75e13cdff P 543ae60e604c41868b2b7ab4a435673b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:05:05.000393 13377 log.cc:1196] T 58370956386b406eaf3f8db75e13cdff P 543ae60e604c41868b2b7ab4a435673b: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-2/wal/wals/58370956386b406eaf3f8db75e13cdff
I20260808 06:05:05.000658 13377 ts_tablet_manager.cc:1939] T 58370956386b406eaf3f8db75e13cdff P 543ae60e604c41868b2b7ab4a435673b: Deleting consensus metadata
I20260808 06:05:05.002858 13374 tablet_replica.cc:330] T 58370956386b406eaf3f8db75e13cdff P 1f756da4d21243bfbc204b1f2950800b: stopping tablet replica
I20260808 06:05:05.003249 13374 raft_consensus.cc:2229] T 58370956386b406eaf3f8db75e13cdff P 1f756da4d21243bfbc204b1f2950800b [term 1 LEADER]: Raft consensus shutting down.
I20260808 06:05:05.003682 13374 raft_consensus.cc:2258] T 58370956386b406eaf3f8db75e13cdff P 1f756da4d21243bfbc204b1f2950800b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:05:05.005208 13374 ts_tablet_manager.cc:1905] T 58370956386b406eaf3f8db75e13cdff P 1f756da4d21243bfbc204b1f2950800b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 06:05:05.006625 13122 connection.cc:535] server connection from 127.0.0.1:53112 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 06:05:05.006990 12724 connection.cc:535] server connection from 127.0.0.1:46298 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 06:05:05.009390 13374 ts_tablet_manager.cc:1918] T 58370956386b406eaf3f8db75e13cdff P 1f756da4d21243bfbc204b1f2950800b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:05:05.009555 13374 log.cc:1196] T 58370956386b406eaf3f8db75e13cdff P 1f756da4d21243bfbc204b1f2950800b: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-1/wal/wals/58370956386b406eaf3f8db75e13cdff
I20260808 06:05:05.009934 13374 ts_tablet_manager.cc:1939] T 58370956386b406eaf3f8db75e13cdff P 1f756da4d21243bfbc204b1f2950800b: Deleting consensus metadata
W20260808 06:05:05.011412 12928 connection.cc:441] server connection from 127.0.0.1:36164 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36164 (request call id 1) could send its response
I20260808 06:05:05.013245 12524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.987375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59608 (local address 127.1.151.189:44883)
0808 06:05:04.987516 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:04.987520 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:05.002568 (+ 15048us) server_negotiation.cc:407] Connection header received
0808 06:05:05.002689 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:05.002692 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:05.002745 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:05.002817 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:05.003863 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:05.004515 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:05.005754 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:05.006070 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:05.006174 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:05.008142 (+ 1968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:05.008146 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:05.008149 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:05.008177 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:05.009896 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:05.010428 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:05.010432 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:05.010434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:05.010528 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:05.010842 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:05.010846 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:05.010848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:05.011114 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:05.011423 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:05.012932 (+ 1509us) server_negotiation.cc:299] Negotiation successful
0808 06:05:05.013075 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 06:05:05.022085 13312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:05.010753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59622 (local address 127.1.151.189:44883)
0808 06:05:05.010912 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:05.010916 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:05.011950 (+ 1034us) server_negotiation.cc:407] Connection header received
0808 06:05:05.012042 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:05.012045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:05.012090 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:05.012168 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:05.013579 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:05.014342 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:05.015430 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:05.015753 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:05.015821 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:05.017524 (+ 1703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:05.017528 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:05.017531 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:05.017555 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:05.019344 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:05.019861 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:05.019864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:05.019866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:05.019951 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:05.020259 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:05.020262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:05.020264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:05.020716 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:05.021749 (+ 1033us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:05.021822 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 06:05:05.021932 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 06:05:05.022187 13316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:04.987420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41116 (local address 127.1.151.188:33465)
0808 06:05:04.987612 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:04.987616 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:05.012197 (+ 24581us) server_negotiation.cc:407] Connection header received
0808 06:05:05.012245 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:05.012248 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:05.012300 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:05.012366 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:05.013405 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:05.014238 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:05.015466 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:05.015833 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:05.015924 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:05.017572 (+ 1648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:05.017576 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:05.017578 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:05.017605 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:05.020098 (+ 2493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:05.020848 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:05.020851 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:05.020853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:05.020933 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:05.021237 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:05.021241 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:05.021242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:05.021438 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:05.021560 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:05.021925 (+ 365us) server_negotiation.cc:299] Negotiation successful
0808 06:05:05.022030 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260808 06:05:05.023459 12899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:05.011372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41120 (local address 127.1.151.188:33465)
0808 06:05:05.011492 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:05.011497 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:05.012220 (+ 723us) server_negotiation.cc:407] Connection header received
0808 06:05:05.012384 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:05.012388 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:05.012446 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:05.012500 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:05.015745 (+ 3245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:05.016831 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:05.017951 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:05.018423 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:05.018521 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:05.020029 (+ 1508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:05.020033 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:05.020036 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:05.020074 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:05.021572 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:05.022252 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:05.022255 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:05.022257 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:05.022338 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:05.022625 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:05.022628 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:05.022630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:05.022814 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:05.022906 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:05.023237 (+ 331us) server_negotiation.cc:299] Negotiation successful
0808 06:05:05.023322 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":51,"spinlock_wait_cycles":2304}
I20260808 06:05:05.973529 13390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:05.973005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35293 (local address 127.1.151.129:57131)
0808 06:05:05.973321 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:05.973415 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:35293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":80,"threads_started":1}
W20260808 06:05:05.973858 12892 heartbeater.cc:644] Failed to heartbeat to 127.1.151.190:35293 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.151.190:35293: connect: Connection refused (error 111)
I20260808 06:05:05.979236 13392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:05.978841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35293 (local address 127.1.151.131:56001)
0808 06:05:05.979051 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:05.979135 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:35293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":71,"threads_started":1}
W20260808 06:05:05.979598 13293 heartbeater.cc:644] Failed to heartbeat to 127.1.151.190:35293 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.151.190:35293: connect: Connection refused (error 111)
I20260808 06:05:05.980733 13393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:05.980375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35293 (local address 127.1.151.130:53823)
0808 06:05:05.980581 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:05.980653 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:35293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":70,"threads_started":1}
W20260808 06:05:05.980968 13089 heartbeater.cc:644] Failed to heartbeat to 127.1.151.190:35293 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.151.190:35293: connect: Connection refused (error 111)
I20260808 06:05:06.485906 13394 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88cc958608974ee987b790f318d04d52)
I20260808 06:05:06.486042 13395 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88cc958608974ee987b790f318d04d52)
I20260808 06:05:06.486127 13394 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: false } } peers { permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 33465 } attrs { promote: false } }
I20260808 06:05:06.486275 13395 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: false } } peers { permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 33465 } attrs { promote: false } }
I20260808 06:05:06.487192 13394 leader_election.cc:288] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88cc958608974ee987b790f318d04d52 (127.1.151.190:35293), 4275957a7be24a9cb4d20a46934a9d1f (127.1.151.188:33465)
I20260808 06:05:06.487201 13395 leader_election.cc:288] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88cc958608974ee987b790f318d04d52 (127.1.151.190:35293), 81fbba8a10f54078bade0f78cc102007 (127.1.151.189:44883)
I20260808 06:05:06.487808 13396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:06.487339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35293 (local address 127.0.0.1:58282)
0808 06:05:06.487606 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:06.487701 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:35293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":73,"threads_started":1}
W20260808 06:05:06.488081 12304 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.151.190:35293: connect: Connection refused (error 111)
I20260808 06:05:06.488253 13398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:06.487711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35293 (local address 127.0.0.1:58284)
0808 06:05:06.488095 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:06.488167 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:35293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":69,"threads_started":1}
W20260808 06:05:06.488425 12596 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.151.190:35293: connect: Connection refused (error 111)
I20260808 06:05:06.489012 13396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:06.488727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35293 (local address 127.0.0.1:58296)
0808 06:05:06.488855 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:06.488900 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:35293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260808 06:05:06.489112 12304 leader_election.cc:334] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 88cc958608974ee987b790f318d04d52 (127.1.151.190:35293): Network error: Client connection negotiation failed: client connection to 127.1.151.190:35293: connect: Connection refused (error 111)
I20260808 06:05:06.489315 13398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:06.489032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35293 (local address 127.0.0.1:58312)
0808 06:05:06.489199 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:06.489250 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:35293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
W20260808 06:05:06.489470 12596 leader_election.cc:334] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 88cc958608974ee987b790f318d04d52 (127.1.151.190:35293): Network error: Client connection negotiation failed: client connection to 127.1.151.190:35293: connect: Connection refused (error 111)
I20260808 06:05:06.492034 13405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:06.487632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59624 (local address 127.1.151.189:44883)
0808 06:05:06.487982 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:06.487986 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:06.488181 (+ 195us) server_negotiation.cc:407] Connection header received
0808 06:05:06.488229 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:06.488232 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:06.488274 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:06.488344 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:05:06.488711 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:06.489473 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:06.490822 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:06.491402 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:06.491507 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:06.491569 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:06.491873 (+ 304us) server_negotiation.cc:299] Negotiation successful
0808 06:05:06.491942 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":54,"threads_started":1}
I20260808 06:05:06.492039 13406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:06.487384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.189:44883 (local address 127.0.0.1:59624)
0808 06:05:06.487986 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:06.487999 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:06.488089 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:06.488462 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:06.488465 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:06.488484 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:05:06.488585 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:06.488593 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:06.489590 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:06.489593 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:06.490702 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:06.490711 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:06.491528 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:06.491532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:06.491704 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:06.491731 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 06:05:06.491826 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 06:05:06.491900 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":68,"threads_started":1}
I20260808 06:05:06.492512 12350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4275957a7be24a9cb4d20a46934a9d1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "81fbba8a10f54078bade0f78cc102007" is_pre_election: true
I20260808 06:05:06.492729 12350 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4275957a7be24a9cb4d20a46934a9d1f in term 1.
I20260808 06:05:06.492789 13407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:06.487910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41136 (local address 127.1.151.188:33465)
0808 06:05:06.488106 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:06.488110 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:06.488125 (+ 15us) server_negotiation.cc:407] Connection header received
0808 06:05:06.488174 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:06.488177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:06.488220 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:06.488966 (+ 746us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:05:06.489316 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:06.490305 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:06.491571 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:06.492290 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:06.492378 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:06.492428 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:06.492616 (+ 188us) server_negotiation.cc:299] Negotiation successful
0808 06:05:06.492689 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":55,"threads_started":1}
I20260808 06:05:06.492825 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:06.487463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:33465 (local address 127.0.0.1:41136)
0808 06:05:06.487631 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:06.487646 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:06.487750 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:06.488810 (+ 1060us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:06.488813 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:06.488834 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:05:06.489172 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:06.489183 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:06.490404 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:06.490406 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:06.491469 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:06.491478 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:06.492415 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:06.492418 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:06.492504 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:06.492527 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 06:05:06.492607 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 06:05:06.492677 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":45,"threads_started":1}
I20260808 06:05:06.493081 12599 leader_election.cc:302] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4275957a7be24a9cb4d20a46934a9d1f, 81fbba8a10f54078bade0f78cc102007; no voters: 88cc958608974ee987b790f318d04d52
I20260808 06:05:06.493291 13395 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 06:05:06.493274 12626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "81fbba8a10f54078bade0f78cc102007" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "4275957a7be24a9cb4d20a46934a9d1f" is_pre_election: true
I20260808 06:05:06.493363 13395 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 88cc958608974ee987b790f318d04d52)
I20260808 06:05:06.493424 13395 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [term 1 FOLLOWER]: Advancing to term 2
I20260808 06:05:06.494498 13395 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: false } } peers { permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 33465 } attrs { promote: false } }
I20260808 06:05:06.494735 12626 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 81fbba8a10f54078bade0f78cc102007 in current term 2: Already voted for candidate 4275957a7be24a9cb4d20a46934a9d1f in this term.
I20260808 06:05:06.494843 13395 leader_election.cc:288] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [CANDIDATE]: Term 2 election: Requested vote from peers 88cc958608974ee987b790f318d04d52 (127.1.151.190:35293), 81fbba8a10f54078bade0f78cc102007 (127.1.151.189:44883)
I20260808 06:05:06.495137 13406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:06.494952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35293 (local address 127.0.0.1:58324)
0808 06:05:06.495040 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:06.495086 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:35293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 06:05:06.495374 12350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4275957a7be24a9cb4d20a46934a9d1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "81fbba8a10f54078bade0f78cc102007"
I20260808 06:05:06.495415 12307 leader_election.cc:302] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 81fbba8a10f54078bade0f78cc102007; no voters: 4275957a7be24a9cb4d20a46934a9d1f, 88cc958608974ee987b790f318d04d52
I20260808 06:05:06.495488 12350 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [term 1 FOLLOWER]: Advancing to term 2
I20260808 06:05:06.495858 13406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:06.495663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35293 (local address 127.0.0.1:58330)
0808 06:05:06.495754 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:06.495803 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:35293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 06:05:06.495955 12596 leader_election.cc:334] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 88cc958608974ee987b790f318d04d52 (127.1.151.190:35293): Network error: Client connection negotiation failed: client connection to 127.1.151.190:35293: connect: Connection refused (error 111)
I20260808 06:05:06.496620 12350 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4275957a7be24a9cb4d20a46934a9d1f in term 2.
I20260808 06:05:06.496726 13394 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 06:05:06.496877 12599 leader_election.cc:302] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4275957a7be24a9cb4d20a46934a9d1f, 81fbba8a10f54078bade0f78cc102007; no voters: 88cc958608974ee987b790f318d04d52
I20260808 06:05:06.497026 13395 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [term 2 FOLLOWER]: Leader election won for term 2
I20260808 06:05:06.497213 13395 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [term 2 LEADER]: Becoming Leader. State: Replica: 4275957a7be24a9cb4d20a46934a9d1f, State: Running, Role: LEADER
I20260808 06:05:06.497458 13395 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: false } } peers { permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 33465 } attrs { promote: false } }
I20260808 06:05:06.498440 13411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4275957a7be24a9cb4d20a46934a9d1f. Latest consensus state: current_term: 2 leader_uuid: "4275957a7be24a9cb4d20a46934a9d1f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: false } } peers { permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 33465 } attrs { promote: false } } }
I20260808 06:05:06.498557 13411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [sys.catalog]: This master's current role is: LEADER
I20260808 06:05:06.498852 13413 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:05:06.501163 13413 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:05:06.501528 13413 catalog_manager.cc:1259] Loaded cluster ID: 34f12aa373e04701a013720e69244e25
I20260808 06:05:06.501577 13413 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:05:06.502077 13413 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:05:06.502465 13413 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f: Loaded TSK: 0
I20260808 06:05:06.503086 13413 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 06:05:06.528926 12620 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41120:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 06:05:06.532270 12620 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 06:05:06.535089 13406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:06.534886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35293 (local address 127.0.0.1:58350)
0808 06:05:06.534979 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:06.535025 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:35293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260808 06:05:06.535157 12350 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [term 2 FOLLOWER]: Refusing update from remote peer 4275957a7be24a9cb4d20a46934a9d1f: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260808 06:05:06.535637 13411 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260808 06:05:06.536015 13406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:06.535804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35293 (local address 127.0.0.1:58352)
0808 06:05:06.535913 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:06.535962 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:35293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 06:05:06.536262 12596 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f -> Peer 88cc958608974ee987b790f318d04d52 (127.1.151.190:35293): Couldn't send request to peer 88cc958608974ee987b790f318d04d52. Status: Network error: Client connection negotiation failed: client connection to 127.1.151.190:35293: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 06:05:06.537248 13394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4275957a7be24a9cb4d20a46934a9d1f. Latest consensus state: current_term: 2 leader_uuid: "4275957a7be24a9cb4d20a46934a9d1f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: false } } peers { permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 33465 } attrs { promote: false } } }
I20260808 06:05:06.537380 13394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 06:05:06.538380 13395 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "4275957a7be24a9cb4d20a46934a9d1f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: false } } peers { permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 33465 } attrs { promote: false } } }
I20260808 06:05:06.538483 13395 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f [sys.catalog]: This master's current role is: LEADER
I20260808 06:05:06.538966 13415 mvcc.cc:205] Tried to move back new op lower bound from 7316148660363292672 to 7316148660215685120. Current Snapshot: MvccSnapshot[applied={T|T < 7316148660363292672}]
I20260808 06:05:06.539136 13394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "4275957a7be24a9cb4d20a46934a9d1f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "88cc958608974ee987b790f318d04d52" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 35293 } } peers { permanent_uuid: "81fbba8a10f54078bade0f78cc102007" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 44883 } attrs { promote: false } } peers { permanent_uuid: "4275957a7be24a9cb4d20a46934a9d1f" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 33465 } attrs { promote: false } } }
I20260808 06:05:06.539251 13394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81fbba8a10f54078bade0f78cc102007 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 06:05:06.548847 13428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:06.544862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53122 (local address 127.1.151.131:33501)
0808 06:05:06.545096 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:06.545100 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:06.545418 (+ 318us) server_negotiation.cc:407] Connection header received
0808 06:05:06.545477 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:06.545480 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:06.545543 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:06.545618 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:05:06.546090 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:06.546634 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:06.547921 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:06.548336 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:06.548429 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:06.548492 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:06.548660 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 06:05:06.548731 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":75,"threads_started":1}
I20260808 06:05:06.548840 13429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:06.544690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:33501 (local address 127.0.0.1:53122)
0808 06:05:06.545246 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:06.545257 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:06.545345 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:06.545830 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:06.545832 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:06.545846 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:05:06.545937 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:06.545941 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:06.546738 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:06.546741 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:06.547828 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:06.547834 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:06.548440 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:06.548443 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:06.548537 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:06.548569 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 06:05:06.548634 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 06:05:06.548703 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":333,"thread_start_us":98,"threads_started":1}
I20260808 06:05:06.549444 13406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:06.544140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:38501 (local address 127.0.0.1:36178)
0808 06:05:06.544552 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:06.544566 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:06.544647 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:06.545873 (+ 1226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:06.545875 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:06.545893 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:05:06.545977 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:06.545983 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:06.547062 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:06.547065 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:06.548108 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:06.548118 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:06.548972 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:06.548976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:06.549139 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:06.549205 (+ 66us) client_negotiation.cc:769] Sending connection context
0808 06:05:06.549261 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 06:05:06.549329 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":54}
I20260808 06:05:06.549525 13427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:06.544867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46308 (local address 127.1.151.129:38287)
0808 06:05:06.545096 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:06.545101 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:06.545116 (+ 15us) server_negotiation.cc:407] Connection header received
0808 06:05:06.545171 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:06.545174 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:06.545224 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:06.545310 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:05:06.545629 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:06.546394 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:06.548199 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:06.548837 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:06.549102 (+ 265us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:06.549164 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:06.549329 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 06:05:06.549409 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":63,"threads_started":1}
I20260808 06:05:06.549539 13398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:06.544685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:38287 (local address 127.0.0.1:46308)
0808 06:05:06.544779 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:06.544794 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:06.544875 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:06.545336 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:06.545338 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:06.545354 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:05:06.545457 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:06.545463 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:06.546516 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:06.546526 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:06.547712 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:06.547720 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:06.548972 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:06.548976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:06.549205 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:06.549231 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 06:05:06.549306 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 06:05:06.549386 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":88}
I20260808 06:05:06.549683 13187 tablet_service.cc:1467] Processing CreateTablet for tablet 9320afe9350941aeb5c01b736ee0cfa9 (DEFAULT_TABLE table=test-table [id=c59b994a17c743a4b06e572a42d7619c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:05:06.549973 13187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9320afe9350941aeb5c01b736ee0cfa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:05:06.549984 13425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:06.544326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36178 (local address 127.1.151.130:38501)
0808 06:05:06.545640 (+ 1314us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:06.545645 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:06.545658 (+ 13us) server_negotiation.cc:407] Connection header received
0808 06:05:06.545705 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:06.545707 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:06.545753 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:06.546089 (+ 336us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:05:06.546227 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:06.546928 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:06.548252 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:06.548838 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:06.549688 (+ 850us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:06.549756 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:06.549791 (+ 35us) server_negotiation.cc:299] Negotiation successful
0808 06:05:06.549862 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1238,"thread_start_us":80,"threads_started":1}
I20260808 06:05:06.550091 12794 tablet_service.cc:1467] Processing CreateTablet for tablet 9320afe9350941aeb5c01b736ee0cfa9 (DEFAULT_TABLE table=test-table [id=c59b994a17c743a4b06e572a42d7619c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:05:06.550379 12794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9320afe9350941aeb5c01b736ee0cfa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:05:06.550487 12995 tablet_service.cc:1467] Processing CreateTablet for tablet 9320afe9350941aeb5c01b736ee0cfa9 (DEFAULT_TABLE table=test-table [id=c59b994a17c743a4b06e572a42d7619c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:05:06.550786 12995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9320afe9350941aeb5c01b736ee0cfa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:05:06.552577 13430 tablet_bootstrap.cc:492] T 9320afe9350941aeb5c01b736ee0cfa9 P 543ae60e604c41868b2b7ab4a435673b: Bootstrap starting.
I20260808 06:05:06.553205 13431 tablet_bootstrap.cc:492] T 9320afe9350941aeb5c01b736ee0cfa9 P a67dd2b3f83143dd958e5cb489a649c2: Bootstrap starting.
I20260808 06:05:06.553524 13430 tablet_bootstrap.cc:654] T 9320afe9350941aeb5c01b736ee0cfa9 P 543ae60e604c41868b2b7ab4a435673b: Neither blocks nor log segments found. Creating new log.
I20260808 06:05:06.554000 13431 tablet_bootstrap.cc:654] T 9320afe9350941aeb5c01b736ee0cfa9 P a67dd2b3f83143dd958e5cb489a649c2: Neither blocks nor log segments found. Creating new log.
I20260808 06:05:06.554080 13432 tablet_bootstrap.cc:492] T 9320afe9350941aeb5c01b736ee0cfa9 P 1f756da4d21243bfbc204b1f2950800b: Bootstrap starting.
I20260808 06:05:06.554630 13430 tablet_bootstrap.cc:492] T 9320afe9350941aeb5c01b736ee0cfa9 P 543ae60e604c41868b2b7ab4a435673b: No bootstrap required, opened a new log
I20260808 06:05:06.554723 13430 ts_tablet_manager.cc:1397] T 9320afe9350941aeb5c01b736ee0cfa9 P 543ae60e604c41868b2b7ab4a435673b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 06:05:06.555107 13430 raft_consensus.cc:348] T 9320afe9350941aeb5c01b736ee0cfa9 P 543ae60e604c41868b2b7ab4a435673b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f756da4d21243bfbc204b1f2950800b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 38501 } } peers { permanent_uuid: "a67dd2b3f83143dd958e5cb489a649c2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 38287 } } peers { permanent_uuid: "543ae60e604c41868b2b7ab4a435673b" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33501 } }
I20260808 06:05:06.555186 13431 tablet_bootstrap.cc:492] T 9320afe9350941aeb5c01b736ee0cfa9 P a67dd2b3f83143dd958e5cb489a649c2: No bootstrap required, opened a new log
I20260808 06:05:06.555228 13430 raft_consensus.cc:374] T 9320afe9350941aeb5c01b736ee0cfa9 P 543ae60e604c41868b2b7ab4a435673b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:05:06.555259 13431 ts_tablet_manager.cc:1397] T 9320afe9350941aeb5c01b736ee0cfa9 P a67dd2b3f83143dd958e5cb489a649c2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 06:05:06.555271 13430 raft_consensus.cc:729] T 9320afe9350941aeb5c01b736ee0cfa9 P 543ae60e604c41868b2b7ab4a435673b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 543ae60e604c41868b2b7ab4a435673b, State: Initialized, Role: FOLLOWER
I20260808 06:05:06.555398 13430 consensus_queue.cc:260] T 9320afe9350941aeb5c01b736ee0cfa9 P 543ae60e604c41868b2b7ab4a435673b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f756da4d21243bfbc204b1f2950800b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 38501 } } peers { permanent_uuid: "a67dd2b3f83143dd958e5cb489a649c2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 38287 } } peers { permanent_uuid: "543ae60e604c41868b2b7ab4a435673b" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33501 } }
I20260808 06:05:06.555703 13430 ts_tablet_manager.cc:1428] T 9320afe9350941aeb5c01b736ee0cfa9 P 543ae60e604c41868b2b7ab4a435673b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 06:05:06.555660 13431 raft_consensus.cc:348] T 9320afe9350941aeb5c01b736ee0cfa9 P a67dd2b3f83143dd958e5cb489a649c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f756da4d21243bfbc204b1f2950800b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 38501 } } peers { permanent_uuid: "a67dd2b3f83143dd958e5cb489a649c2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 38287 } } peers { permanent_uuid: "543ae60e604c41868b2b7ab4a435673b" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33501 } }
I20260808 06:05:06.555758 13431 raft_consensus.cc:374] T 9320afe9350941aeb5c01b736ee0cfa9 P a67dd2b3f83143dd958e5cb489a649c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:05:06.555788 13431 raft_consensus.cc:729] T 9320afe9350941aeb5c01b736ee0cfa9 P a67dd2b3f83143dd958e5cb489a649c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a67dd2b3f83143dd958e5cb489a649c2, State: Initialized, Role: FOLLOWER
I20260808 06:05:06.555922 13431 consensus_queue.cc:260] T 9320afe9350941aeb5c01b736ee0cfa9 P a67dd2b3f83143dd958e5cb489a649c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f756da4d21243bfbc204b1f2950800b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 38501 } } peers { permanent_uuid: "a67dd2b3f83143dd958e5cb489a649c2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 38287 } } peers { permanent_uuid: "543ae60e604c41868b2b7ab4a435673b" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33501 } }
I20260808 06:05:06.556257 13431 ts_tablet_manager.cc:1428] T 9320afe9350941aeb5c01b736ee0cfa9 P a67dd2b3f83143dd958e5cb489a649c2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 06:05:06.556814 13432 tablet_bootstrap.cc:654] T 9320afe9350941aeb5c01b736ee0cfa9 P 1f756da4d21243bfbc204b1f2950800b: Neither blocks nor log segments found. Creating new log.
I20260808 06:05:06.557531 13439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:06.556026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35293 (local address 127.1.151.131:51211)
0808 06:05:06.557371 (+ 1345us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:06.557426 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:35293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1279,"thread_start_us":83,"threads_started":1}
I20260808 06:05:06.557978 13292 heartbeater.cc:503] Master 127.1.151.188:33465 was elected leader, sending a full tablet report...
I20260808 06:05:06.558306 13432 tablet_bootstrap.cc:492] T 9320afe9350941aeb5c01b736ee0cfa9 P 1f756da4d21243bfbc204b1f2950800b: No bootstrap required, opened a new log
I20260808 06:05:06.558398 13432 ts_tablet_manager.cc:1397] T 9320afe9350941aeb5c01b736ee0cfa9 P 1f756da4d21243bfbc204b1f2950800b: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 06:05:06.558849 13432 raft_consensus.cc:348] T 9320afe9350941aeb5c01b736ee0cfa9 P 1f756da4d21243bfbc204b1f2950800b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f756da4d21243bfbc204b1f2950800b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 38501 } } peers { permanent_uuid: "a67dd2b3f83143dd958e5cb489a649c2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 38287 } } peers { permanent_uuid: "543ae60e604c41868b2b7ab4a435673b" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33501 } }
I20260808 06:05:06.558969 13432 raft_consensus.cc:374] T 9320afe9350941aeb5c01b736ee0cfa9 P 1f756da4d21243bfbc204b1f2950800b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:05:06.559015 13432 raft_consensus.cc:729] T 9320afe9350941aeb5c01b736ee0cfa9 P 1f756da4d21243bfbc204b1f2950800b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f756da4d21243bfbc204b1f2950800b, State: Initialized, Role: FOLLOWER
I20260808 06:05:06.559140 13432 consensus_queue.cc:260] T 9320afe9350941aeb5c01b736ee0cfa9 P 1f756da4d21243bfbc204b1f2950800b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f756da4d21243bfbc204b1f2950800b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 38501 } } peers { permanent_uuid: "a67dd2b3f83143dd958e5cb489a649c2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 38287 } } peers { permanent_uuid: "543ae60e604c41868b2b7ab4a435673b" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33501 } }
I20260808 06:05:06.559439 13432 ts_tablet_manager.cc:1428] T 9320afe9350941aeb5c01b736ee0cfa9 P 1f756da4d21243bfbc204b1f2950800b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 06:05:06.560424 13444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:06.559947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35293 (local address 127.1.151.130:33189)
0808 06:05:06.560255 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:06.560311 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:35293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":97,"threads_started":1}
I20260808 06:05:06.560830 13088 heartbeater.cc:503] Master 127.1.151.188:33465 was elected leader, sending a full tablet report...
I20260808 06:05:06.561033 13442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:06.557448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35293 (local address 127.1.151.129:50277)
0808 06:05:06.560873 (+ 3425us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:06.560931 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:35293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3354,"thread_start_us":117,"threads_started":1}
I20260808 06:05:06.561339 13444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:06.560908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35293 (local address 127.1.151.130:36865)
0808 06:05:06.561057 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:06.561104 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:35293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260808 06:05:06.561342 13439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:06.558258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35293 (local address 127.1.151.131:46607)
0808 06:05:06.561224 (+ 2966us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:06.561279 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:35293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2897}
I20260808 06:05:06.562263 12889 heartbeater.cc:503] Master 127.1.151.188:33465 was elected leader, sending a full tablet report...
I20260808 06:05:06.562672 13442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:06.562446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35293 (local address 127.1.151.129:40365)
0808 06:05:06.562582 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:06.562621 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:35293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260808 06:05:06.587069 13091 tablet.cc:2369] T 9320afe9350941aeb5c01b736ee0cfa9 P 1f756da4d21243bfbc204b1f2950800b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 06:05:06.701479 13436 raft_consensus.cc:482] T 9320afe9350941aeb5c01b736ee0cfa9 P 543ae60e604c41868b2b7ab4a435673b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:05:06.701622 13436 raft_consensus.cc:504] T 9320afe9350941aeb5c01b736ee0cfa9 P 543ae60e604c41868b2b7ab4a435673b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f756da4d21243bfbc204b1f2950800b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 38501 } } peers { permanent_uuid: "a67dd2b3f83143dd958e5cb489a649c2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 38287 } } peers { permanent_uuid: "543ae60e604c41868b2b7ab4a435673b" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33501 } }
I20260808 06:05:06.702759 13436 leader_election.cc:288] T 9320afe9350941aeb5c01b736ee0cfa9 P 543ae60e604c41868b2b7ab4a435673b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f756da4d21243bfbc204b1f2950800b (127.1.151.130:38501), a67dd2b3f83143dd958e5cb489a649c2 (127.1.151.129:38287)
I20260808 06:05:06.706588 13427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:06.703089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:56261 (local address 127.1.151.129:38287)
0808 06:05:06.703219 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:06.703223 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:06.703239 (+ 16us) server_negotiation.cc:407] Connection header received
0808 06:05:06.703371 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:06.703375 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:06.703431 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:06.703524 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:05:06.703859 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:06.704442 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:06.705569 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:06.705953 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:06.706056 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:06.706118 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:06.706394 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 06:05:06.706463 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 06:05:06.706595 13425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:06.703089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:58961 (local address 127.1.151.130:38501)
0808 06:05:06.703214 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:06.703219 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:06.703527 (+ 308us) server_negotiation.cc:407] Connection header received
0808 06:05:06.703655 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:06.703659 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:06.703709 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:06.703789 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:05:06.704040 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:06.704603 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:06.705593 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:06.705983 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:06.706092 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:06.706143 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:06.706391 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 06:05:06.706463 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 06:05:06.706595 13439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:06.702988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:38287 (local address 127.1.151.131:56261)
0808 06:05:06.703129 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:06.703145 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:06.703256 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:06.703587 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:06.703591 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:06.703614 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:05:06.703748 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:06.703755 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:06.704552 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:06.704556 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:06.705427 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:06.705438 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:06.706076 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:06.706080 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:06.706213 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:06.706288 (+ 75us) client_negotiation.cc:769] Sending connection context
0808 06:05:06.706352 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 06:05:06.706443 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":34}
I20260808 06:05:06.706619 13447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:06.702988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:38501 (local address 127.1.151.131:58961)
0808 06:05:06.703432 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:06.703446 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:06.703543 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:06.703795 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:06.703797 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:06.703814 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:05:06.703930 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:06.703936 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:06.704716 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:06.704718 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:06.705456 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:06.705465 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:06.706095 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:06.706099 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:06.706249 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:06.706293 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 06:05:06.706360 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 06:05:06.706460 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"mutex_wait_us":163,"thread_start_us":142,"threads_started":1}
I20260808 06:05:06.707046 12821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9320afe9350941aeb5c01b736ee0cfa9" candidate_uuid: "543ae60e604c41868b2b7ab4a435673b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a67dd2b3f83143dd958e5cb489a649c2" is_pre_election: true
I20260808 06:05:06.707237 12821 raft_consensus.cc:2454] T 9320afe9350941aeb5c01b736ee0cfa9 P a67dd2b3f83143dd958e5cb489a649c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 543ae60e604c41868b2b7ab4a435673b in term 0.
I20260808 06:05:06.707381 13020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9320afe9350941aeb5c01b736ee0cfa9" candidate_uuid: "543ae60e604c41868b2b7ab4a435673b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f756da4d21243bfbc204b1f2950800b" is_pre_election: true
I20260808 06:05:06.707610 13020 raft_consensus.cc:2454] T 9320afe9350941aeb5c01b736ee0cfa9 P 1f756da4d21243bfbc204b1f2950800b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 543ae60e604c41868b2b7ab4a435673b in term 0.
I20260808 06:05:06.707610 13122 leader_election.cc:302] T 9320afe9350941aeb5c01b736ee0cfa9 P 543ae60e604c41868b2b7ab4a435673b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 543ae60e604c41868b2b7ab4a435673b, a67dd2b3f83143dd958e5cb489a649c2; no voters:
I20260808 06:05:06.707871 13436 raft_consensus.cc:2781] T 9320afe9350941aeb5c01b736ee0cfa9 P 543ae60e604c41868b2b7ab4a435673b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 06:05:06.707934 13436 raft_consensus.cc:482] T 9320afe9350941aeb5c01b736ee0cfa9 P 543ae60e604c41868b2b7ab4a435673b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 06:05:06.707973 13436 raft_consensus.cc:3037] T 9320afe9350941aeb5c01b736ee0cfa9 P 543ae60e604c41868b2b7ab4a435673b [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:05:06.709010 13436 raft_consensus.cc:504] T 9320afe9350941aeb5c01b736ee0cfa9 P 543ae60e604c41868b2b7ab4a435673b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f756da4d21243bfbc204b1f2950800b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 38501 } } peers { permanent_uuid: "a67dd2b3f83143dd958e5cb489a649c2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 38287 } } peers { permanent_uuid: "543ae60e604c41868b2b7ab4a435673b" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33501 } }
I20260808 06:05:06.709317 13436 leader_election.cc:288] T 9320afe9350941aeb5c01b736ee0cfa9 P 543ae60e604c41868b2b7ab4a435673b [CANDIDATE]: Term 1 election: Requested vote from peers 1f756da4d21243bfbc204b1f2950800b (127.1.151.130:38501), a67dd2b3f83143dd958e5cb489a649c2 (127.1.151.129:38287)
I20260808 06:05:06.709679 13020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9320afe9350941aeb5c01b736ee0cfa9" candidate_uuid: "543ae60e604c41868b2b7ab4a435673b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f756da4d21243bfbc204b1f2950800b"
I20260808 06:05:06.709712 12821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9320afe9350941aeb5c01b736ee0cfa9" candidate_uuid: "543ae60e604c41868b2b7ab4a435673b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a67dd2b3f83143dd958e5cb489a649c2"
I20260808 06:05:06.709808 13020 raft_consensus.cc:3037] T 9320afe9350941aeb5c01b736ee0cfa9 P 1f756da4d21243bfbc204b1f2950800b [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:05:06.709808 12821 raft_consensus.cc:3037] T 9320afe9350941aeb5c01b736ee0cfa9 P a67dd2b3f83143dd958e5cb489a649c2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:05:06.710944 12821 raft_consensus.cc:2454] T 9320afe9350941aeb5c01b736ee0cfa9 P a67dd2b3f83143dd958e5cb489a649c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 543ae60e604c41868b2b7ab4a435673b in term 1.
I20260808 06:05:06.710973 13020 raft_consensus.cc:2454] T 9320afe9350941aeb5c01b736ee0cfa9 P 1f756da4d21243bfbc204b1f2950800b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 543ae60e604c41868b2b7ab4a435673b in term 1.
I20260808 06:05:06.711231 13118 leader_election.cc:302] T 9320afe9350941aeb5c01b736ee0cfa9 P 543ae60e604c41868b2b7ab4a435673b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f756da4d21243bfbc204b1f2950800b, 543ae60e604c41868b2b7ab4a435673b; no voters:
I20260808 06:05:06.711386 13436 raft_consensus.cc:2781] T 9320afe9350941aeb5c01b736ee0cfa9 P 543ae60e604c41868b2b7ab4a435673b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:05:06.711585 13436 raft_consensus.cc:686] T 9320afe9350941aeb5c01b736ee0cfa9 P 543ae60e604c41868b2b7ab4a435673b [term 1 LEADER]: Becoming Leader. State: Replica: 543ae60e604c41868b2b7ab4a435673b, State: Running, Role: LEADER
I20260808 06:05:06.711854 13436 consensus_queue.cc:237] T 9320afe9350941aeb5c01b736ee0cfa9 P 543ae60e604c41868b2b7ab4a435673b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f756da4d21243bfbc204b1f2950800b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 38501 } } peers { permanent_uuid: "a67dd2b3f83143dd958e5cb489a649c2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 38287 } } peers { permanent_uuid: "543ae60e604c41868b2b7ab4a435673b" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33501 } }
I20260808 06:05:06.712368 13447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:06.712152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35293 (local address 127.1.151.131:59677)
0808 06:05:06.712253 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:06.712308 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:35293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 06:05:06.714012 12613 catalog_manager.cc:5495] T 9320afe9350941aeb5c01b736ee0cfa9 P 543ae60e604c41868b2b7ab4a435673b reported cstate change: term changed from 0 to 1, leader changed from <none> to 543ae60e604c41868b2b7ab4a435673b (127.1.151.131). New cstate: current_term: 1 leader_uuid: "543ae60e604c41868b2b7ab4a435673b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f756da4d21243bfbc204b1f2950800b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 38501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a67dd2b3f83143dd958e5cb489a649c2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 38287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "543ae60e604c41868b2b7ab4a435673b" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33501 } health_report { overall_health: HEALTHY } } }
I20260808 06:05:06.735386 13428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:06.732363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53132 (local address 127.1.151.131:33501)
0808 06:05:06.732465 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:06.732468 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:06.732479 (+ 11us) server_negotiation.cc:407] Connection header received
0808 06:05:06.732607 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:06.732610 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:06.732662 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:06.732746 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:05:06.732919 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:06.733426 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:06.734231 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:06.734438 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:06.734512 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:06.734705 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:05:06.735117 (+ 412us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:05:06.735199 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:06.735241 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 06:05:06.735287 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 06:05:06.737877 12821 raft_consensus.cc:1261] T 9320afe9350941aeb5c01b736ee0cfa9 P a67dd2b3f83143dd958e5cb489a649c2 [term 1 FOLLOWER]: Refusing update from remote peer 543ae60e604c41868b2b7ab4a435673b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:05:06.738090 13020 raft_consensus.cc:1261] T 9320afe9350941aeb5c01b736ee0cfa9 P 1f756da4d21243bfbc204b1f2950800b [term 1 FOLLOWER]: Refusing update from remote peer 543ae60e604c41868b2b7ab4a435673b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:05:06.738447 13450 consensus_queue.cc:1035] T 9320afe9350941aeb5c01b736ee0cfa9 P 543ae60e604c41868b2b7ab4a435673b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a67dd2b3f83143dd958e5cb489a649c2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 38287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:05:06.738656 13436 consensus_queue.cc:1035] T 9320afe9350941aeb5c01b736ee0cfa9 P 543ae60e604c41868b2b7ab4a435673b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f756da4d21243bfbc204b1f2950800b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 38501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:05:06.740089 13442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:06.739801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35293 (local address 127.1.151.129:41793)
0808 06:05:06.739957 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:06.740015 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:35293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260808 06:05:06.741490 13444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:06.741156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35293 (local address 127.1.151.130:45421)
0808 06:05:06.741365 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:06.741419 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:35293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144}
I20260808 06:05:06.743273 13447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:06.742961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35293 (local address 127.1.151.131:56451)
0808 06:05:06.743140 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:06.743192 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:35293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260808 06:05:06.744966 13447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:06.744715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35293 (local address 127.1.151.131:42241)
0808 06:05:06.744853 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:06.744906 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:35293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260808 06:05:06.745370 13458 mvcc.cc:205] Tried to move back new op lower bound from 7316148661193736192 to 7316148661093896192. Current Snapshot: MvccSnapshot[applied={T|T < 7316148661193736192}]
I20260808 06:05:06.745651 13442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:06.745208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35293 (local address 127.1.151.129:38381)
0808 06:05:06.745507 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:06.745582 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:35293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227}
I20260808 06:05:06.745766 13444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:06.745536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:35293 (local address 127.1.151.130:47217)
0808 06:05:06.745649 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:06.745700 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:35293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 06:05:06.754338 12613 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:05:06.754473 12613 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:05:06.757972 12613 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4275957a7be24a9cb4d20a46934a9d1f: Sending DeleteTablet for 3 replicas of tablet 9320afe9350941aeb5c01b736ee0cfa9
I20260808 06:05:06.758839 12794 tablet_service.cc:1514] Processing DeleteTablet for tablet 9320afe9350941aeb5c01b736ee0cfa9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:05:06 UTC) from {username='oryx', principal='oryx/127.1.151.188@KRBTEST.COM'} at 127.0.0.1:46308
I20260808 06:05:06.758936 12995 tablet_service.cc:1514] Processing DeleteTablet for tablet 9320afe9350941aeb5c01b736ee0cfa9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:05:06 UTC) from {username='oryx', principal='oryx/127.1.151.188@KRBTEST.COM'} at 127.0.0.1:36178
I20260808 06:05:06.759330 13187 tablet_service.cc:1514] Processing DeleteTablet for tablet 9320afe9350941aeb5c01b736ee0cfa9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:05:06 UTC) from {username='oryx', principal='oryx/127.1.151.188@KRBTEST.COM'} at 127.0.0.1:53122
I20260808 06:05:06.759502 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 12558
I20260808 06:05:06.759701 13472 tablet_replica.cc:330] T 9320afe9350941aeb5c01b736ee0cfa9 P 543ae60e604c41868b2b7ab4a435673b: stopping tablet replica
I20260808 06:05:06.759847 13472 raft_consensus.cc:2229] T 9320afe9350941aeb5c01b736ee0cfa9 P 543ae60e604c41868b2b7ab4a435673b [term 1 LEADER]: Raft consensus shutting down.
I20260808 06:05:06.760185 13472 raft_consensus.cc:2258] T 9320afe9350941aeb5c01b736ee0cfa9 P 543ae60e604c41868b2b7ab4a435673b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:05:06.762779 13472 ts_tablet_manager.cc:1905] T 9320afe9350941aeb5c01b736ee0cfa9 P 543ae60e604c41868b2b7ab4a435673b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:05:06.765579 13471 tablet_replica.cc:330] T 9320afe9350941aeb5c01b736ee0cfa9 P 1f756da4d21243bfbc204b1f2950800b: stopping tablet replica
I20260808 06:05:06.765719 13471 raft_consensus.cc:2229] T 9320afe9350941aeb5c01b736ee0cfa9 P 1f756da4d21243bfbc204b1f2950800b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:05:06.765846 13471 raft_consensus.cc:2258] T 9320afe9350941aeb5c01b736ee0cfa9 P 1f756da4d21243bfbc204b1f2950800b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:05:06.766088 13472 ts_tablet_manager.cc:1918] T 9320afe9350941aeb5c01b736ee0cfa9 P 543ae60e604c41868b2b7ab4a435673b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:05:06.766184 13472 log.cc:1196] T 9320afe9350941aeb5c01b736ee0cfa9 P 543ae60e604c41868b2b7ab4a435673b: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-2/wal/wals/9320afe9350941aeb5c01b736ee0cfa9
I20260808 06:05:06.766475 13472 ts_tablet_manager.cc:1939] T 9320afe9350941aeb5c01b736ee0cfa9 P 543ae60e604c41868b2b7ab4a435673b: Deleting consensus metadata
I20260808 06:05:06.767063 12596 catalog_manager.cc:4841] TS 543ae60e604c41868b2b7ab4a435673b (127.1.151.131:33501): tablet 9320afe9350941aeb5c01b736ee0cfa9 (table test-table [id=c59b994a17c743a4b06e572a42d7619c]) successfully deleted
I20260808 06:05:06.770716 13471 ts_tablet_manager.cc:1905] T 9320afe9350941aeb5c01b736ee0cfa9 P 1f756da4d21243bfbc204b1f2950800b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:05:06.772810 13471 ts_tablet_manager.cc:1918] T 9320afe9350941aeb5c01b736ee0cfa9 P 1f756da4d21243bfbc204b1f2950800b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:05:06.772893 13471 log.cc:1196] T 9320afe9350941aeb5c01b736ee0cfa9 P 1f756da4d21243bfbc204b1f2950800b: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786169070948234-1630-0/minicluster-data/ts-1/wal/wals/9320afe9350941aeb5c01b736ee0cfa9
I20260808 06:05:06.773180 13471 ts_tablet_manager.cc:1939] T 9320afe9350941aeb5c01b736ee0cfa9 P 1f756da4d21243bfbc204b1f2950800b: Deleting consensus metadata
I20260808 06:05:06.773793 12596 catalog_manager.cc:4841] TS 1f756da4d21243bfbc204b1f2950800b (127.1.151.130:38501): tablet 9320afe9350941aeb5c01b736ee0cfa9 (table test-table [id=c59b994a17c743a4b06e572a42d7619c]) successfully deleted
I20260808 06:05:06.775763 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 12901
I20260808 06:05:06.775843 13398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:06.775556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:38287 (local address 127.0.0.1:46316)
0808 06:05:06.775713 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:06.775767 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.129:38287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260808 06:05:06.782358 12599 catalog_manager.cc:4584] TS a67dd2b3f83143dd958e5cb489a649c2 (127.1.151.129:38287): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9320afe9350941aeb5c01b736ee0cfa9: Network error: Client connection negotiation failed: client connection to 127.1.151.129:38287: connect: Connection refused (error 111)
I20260808 06:05:06.788388 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 13095
I20260808 06:05:06.797082 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 12109
I20260808 06:05:06.807025 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 12285
2026-08-08T06:05:06Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5423 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:05:06 dist-test-slave-zj8q krb5kdc[13476](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:05:06 dist-test-slave-zj8q krb5kdc[13476](info): set up 2 sockets
Aug 08 06:05:06 dist-test-slave-zj8q krb5kdc[13476](info): commencing operation
krb5kdc: starting...
W20260808 06:05:07.951300 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:05:08 dist-test-slave-zj8q krb5kdc[13476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169108, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:05:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:05:08Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.151.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.151.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.151.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.190.keytab.
Entry for principal oryx/127.1.151.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.190.keytab.
I20260808 06:05:08.074141 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:41443
--webserver_interface=127.1.151.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.190.keytab
--principal=oryx/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:44679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:41443
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:05:08.183727 13496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:08.183997 13496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:08.184055 13496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:08.188063 13496 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:05:08.188118 13496 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 06:05:08.188141 13496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:08.188166 13496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:08.188197 13496 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:05:08.188223 13496 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:05:08.193312 13496 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:44679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:41443
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.190.keytab
--principal=oryx/127.1.151.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:41443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:05:08.194401 13496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:08.195405 13496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:08.203732 13506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:08.203768 13503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:08.203920 13496 server_base.cc:1034] running on GCE node
W20260808 06:05:08.203768 13504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:08.204649 13496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:08.205750 13496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:08.206912 13496 hybrid_clock.cc:648] HybridClock initialized: now 1786169108206885 us; error 42 us; skew 500 ppm
Aug 08 06:05:08 dist-test-slave-zj8q krb5kdc[13476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169108, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:08.212108 13496 init.cc:377] Logged in from keytab as oryx/127.1.151.190@KRBTEST.COM (short username oryx)
I20260808 06:05:08.212973 13496 webserver.cc:460] Webserver started at http://127.1.151.190:38103/ using document root <none> and password file <none>
I20260808 06:05:08.213523 13496 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:08.213591 13496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:08.213784 13496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:05:08.215862 13496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "f5f0153a8bbb4a819b1bb2eebff8ca4f"
format_stamp: "Formatted at 2026-08-08 06:05:08 on dist-test-slave-zj8q"
server_key: "4d099d7f550c5bb00b0cc7ff175010f2"
server_key_iv: "9b04f39d2b458a436d8afa3f92bf8ad3"
server_key_version: "encryptionkey@0"
I20260808 06:05:08.216413 13496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "f5f0153a8bbb4a819b1bb2eebff8ca4f"
format_stamp: "Formatted at 2026-08-08 06:05:08 on dist-test-slave-zj8q"
server_key: "4d099d7f550c5bb00b0cc7ff175010f2"
server_key_iv: "9b04f39d2b458a436d8afa3f92bf8ad3"
server_key_version: "encryptionkey@0"
I20260808 06:05:08.220112 13496 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260808 06:05:08.222468 13515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:08.223785 13496 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 06:05:08.223906 13496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "f5f0153a8bbb4a819b1bb2eebff8ca4f"
format_stamp: "Formatted at 2026-08-08 06:05:08 on dist-test-slave-zj8q"
server_key: "4d099d7f550c5bb00b0cc7ff175010f2"
server_key_iv: "9b04f39d2b458a436d8afa3f92bf8ad3"
server_key_version: "encryptionkey@0"
I20260808 06:05:08.224002 13496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:08.242578 13496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:08.245458 13496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:05:08.245652 13496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:08.253226 13496 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:41443
I20260808 06:05:08.253284 13605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:41443 every 8 connection(s)
I20260808 06:05:08.254182 13496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:05:08.256994 13606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:05:08.260092 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 13496
I20260808 06:05:08.260200 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:05:08.260464 1630 external_mini_cluster.cc:1428] Setting key 6723b7557f26719a2126edd53d7a3ad8
I20260808 06:05:08.262835 13606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5f0153a8bbb4a819b1bb2eebff8ca4f: Bootstrap starting.
I20260808 06:05:08.265470 13606 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5f0153a8bbb4a819b1bb2eebff8ca4f: Neither blocks nor log segments found. Creating new log.
I20260808 06:05:08.266359 13606 log.cc:824] T 00000000000000000000000000000000 P f5f0153a8bbb4a819b1bb2eebff8ca4f: Log is configured to *not* fsync() on all Append() calls
I20260808 06:05:08.268576 13606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5f0153a8bbb4a819b1bb2eebff8ca4f: No bootstrap required, opened a new log
Aug 08 06:05:08 dist-test-slave-zj8q krb5kdc[13476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169108, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM
I20260808 06:05:08.272622 13606 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5f0153a8bbb4a819b1bb2eebff8ca4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5f0153a8bbb4a819b1bb2eebff8ca4f" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 41443 } }
I20260808 06:05:08.272890 13606 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5f0153a8bbb4a819b1bb2eebff8ca4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:05:08.272953 13606 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5f0153a8bbb4a819b1bb2eebff8ca4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f0153a8bbb4a819b1bb2eebff8ca4f, State: Initialized, Role: FOLLOWER
I20260808 06:05:08.273427 13606 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5f0153a8bbb4a819b1bb2eebff8ca4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5f0153a8bbb4a819b1bb2eebff8ca4f" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 41443 } }
I20260808 06:05:08.273595 13606 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5f0153a8bbb4a819b1bb2eebff8ca4f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:05:08.273665 13606 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5f0153a8bbb4a819b1bb2eebff8ca4f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:05:08.273767 13606 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5f0153a8bbb4a819b1bb2eebff8ca4f [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:05:08.274873 13606 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5f0153a8bbb4a819b1bb2eebff8ca4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5f0153a8bbb4a819b1bb2eebff8ca4f" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 41443 } }
I20260808 06:05:08.275223 13606 leader_election.cc:302] T 00000000000000000000000000000000 P f5f0153a8bbb4a819b1bb2eebff8ca4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5f0153a8bbb4a819b1bb2eebff8ca4f; no voters:
I20260808 06:05:08.275355 13610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:08.261913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47718 (local address 127.1.151.190:41443)
0808 06:05:08.262344 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:08.262363 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:08.262406 (+ 43us) server_negotiation.cc:407] Connection header received
0808 06:05:08.263201 (+ 795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:08.263224 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:08.263612 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:08.263965 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:08.264618 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:08.266104 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:08.267138 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:08.267794 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:08.267954 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:08.270468 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:08.270474 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:08.270490 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:08.270521 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:08.272289 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:08.272684 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:08.272690 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:08.272695 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:08.272826 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:08.273107 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:08.273110 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:08.273111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:08.273579 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:08.273780 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:08.274105 (+ 325us) server_negotiation.cc:299] Negotiation successful
0808 06:05:08.274411 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":109,"threads_started":1}
I20260808 06:05:08.275552 13606 leader_election.cc:288] T 00000000000000000000000000000000 P f5f0153a8bbb4a819b1bb2eebff8ca4f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:05:08.275718 13612 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5f0153a8bbb4a819b1bb2eebff8ca4f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:05:08.275887 13612 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5f0153a8bbb4a819b1bb2eebff8ca4f [term 1 LEADER]: Becoming Leader. State: Replica: f5f0153a8bbb4a819b1bb2eebff8ca4f, State: Running, Role: LEADER
I20260808 06:05:08.276223 13612 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5f0153a8bbb4a819b1bb2eebff8ca4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5f0153a8bbb4a819b1bb2eebff8ca4f" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 41443 } }
I20260808 06:05:08.276800 13606 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5f0153a8bbb4a819b1bb2eebff8ca4f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:05:08.277920 13614 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5f0153a8bbb4a819b1bb2eebff8ca4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5f0153a8bbb4a819b1bb2eebff8ca4f. Latest consensus state: current_term: 1 leader_uuid: "f5f0153a8bbb4a819b1bb2eebff8ca4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5f0153a8bbb4a819b1bb2eebff8ca4f" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 41443 } } }
I20260808 06:05:08.278072 13614 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5f0153a8bbb4a819b1bb2eebff8ca4f [sys.catalog]: This master's current role is: LEADER
I20260808 06:05:08.278401 13613 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5f0153a8bbb4a819b1bb2eebff8ca4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5f0153a8bbb4a819b1bb2eebff8ca4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5f0153a8bbb4a819b1bb2eebff8ca4f" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 41443 } } }
I20260808 06:05:08.278491 13624 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:05:08.278496 13613 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5f0153a8bbb4a819b1bb2eebff8ca4f [sys.catalog]: This master's current role is: LEADER
I20260808 06:05:08.281478 13624 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:05:08.288113 13624 catalog_manager.cc:1347] Generated new cluster ID: 3340973a8488443ea25692eca8df88aa
I20260808 06:05:08.288199 13624 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:05:08.299969 13624 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:05:08.300887 13624 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:05:08.320417 13624 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5f0153a8bbb4a819b1bb2eebff8ca4f: Generated new TSK 0
I20260808 06:05:08.321103 13624 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.151.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.151.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.151.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.129.keytab.
Entry for principal oryx/127.1.151.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.129.keytab.
I20260808 06:05:08.349125 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.129:0
--local_ip_for_outbound_sockets=127.1.151.129
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:41443
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.129.keytab
--principal=oryx/127.1.151.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:44679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:05:08.459115 13645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:08.459362 13645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:08.459422 13645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:08.463305 13645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:08.463359 13645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:08.463411 13645 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 06:05:08.463460 13645 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.129
I20260808 06:05:08.468565 13645 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:44679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.129.keytab
--principal=oryx/127.1.151.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.151.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.151.190:41443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.129
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:05:08.469537 13645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:08.470552 13645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:08.479914 13652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:08.479960 13653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:08.480005 13645 server_base.cc:1034] running on GCE node
W20260808 06:05:08.479928 13655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:08.480522 13645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:08.481112 13645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:08.482223 13645 hybrid_clock.cc:648] HybridClock initialized: now 1786169108482203 us; error 28 us; skew 500 ppm
Aug 08 06:05:08 dist-test-slave-zj8q krb5kdc[13476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169108, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:08.484918 13645 init.cc:377] Logged in from keytab as oryx/127.1.151.129@KRBTEST.COM (short username oryx)
I20260808 06:05:08.485769 13645 webserver.cc:460] Webserver started at http://127.1.151.129:45679/ using document root <none> and password file <none>
I20260808 06:05:08.486336 13645 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:08.486404 13645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:08.486603 13645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:05:08.488579 13645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-0/data/instance:
uuid: "d784a288c65342a8a9b126e4ec8a618f"
format_stamp: "Formatted at 2026-08-08 06:05:08 on dist-test-slave-zj8q"
server_key: "4de76e198b57e28b9b3ea360cb0a06fb"
server_key_iv: "b5462b706778ed172178dca19f95548f"
server_key_version: "encryptionkey@0"
I20260808 06:05:08.489120 13645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance:
uuid: "d784a288c65342a8a9b126e4ec8a618f"
format_stamp: "Formatted at 2026-08-08 06:05:08 on dist-test-slave-zj8q"
server_key: "4de76e198b57e28b9b3ea360cb0a06fb"
server_key_iv: "b5462b706778ed172178dca19f95548f"
server_key_version: "encryptionkey@0"
I20260808 06:05:08.492615 13645 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 06:05:08.494944 13662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:08.496102 13645 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:05:08.496212 13645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-0/wal
uuid: "d784a288c65342a8a9b126e4ec8a618f"
format_stamp: "Formatted at 2026-08-08 06:05:08 on dist-test-slave-zj8q"
server_key: "4de76e198b57e28b9b3ea360cb0a06fb"
server_key_iv: "b5462b706778ed172178dca19f95548f"
server_key_version: "encryptionkey@0"
I20260808 06:05:08.496299 13645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:08.508219 13645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:08.510830 13645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:05:08.510977 13645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:08.512290 13645 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:05:08.512341 13645 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:08.512390 13645 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:05:08.512419 13645 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 06:05:08 dist-test-slave-zj8q krb5kdc[13476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169108, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM
I20260808 06:05:08.522208 13645 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.129:35179
I20260808 06:05:08.522313 13850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.129:35179 every 8 connection(s)
I20260808 06:05:08.523195 13645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
I20260808 06:05:08.525909 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 13645
I20260808 06:05:08.526041 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance
I20260808 06:05:08.526050 13610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:08.513103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:43585 (local address 127.1.151.190:41443)
0808 06:05:08.513225 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:08.513229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:08.514246 (+ 1017us) server_negotiation.cc:407] Connection header received
0808 06:05:08.515264 (+ 1018us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:08.515269 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:08.515334 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:08.515427 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:08.516939 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:08.517759 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:08.519509 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:08.519877 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:08.519993 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:08.522708 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:08.522713 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:08.522716 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:08.522745 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:08.524124 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:08.524596 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:08.524600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:08.524601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:08.524685 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:08.525001 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:08.525004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:08.525005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:08.525184 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:08.525269 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:08.525759 (+ 490us) server_negotiation.cc:299] Negotiation successful
0808 06:05:08.525886 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 06:05:08.526371 1630 external_mini_cluster.cc:1428] Setting key 67cd4433a17dc8a1b114894ae1202cd1
I20260808 06:05:08.526868 13680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:08.513595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:41443 (local address 127.1.151.129:43585)
0808 06:05:08.514080 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:08.514120 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:08.515030 (+ 910us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:08.515708 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:08.515716 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:08.516175 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:08.516756 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:08.516774 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:08.517898 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:08.517902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:08.519369 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:08.519382 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:08.520017 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:08.520020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:08.520172 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:08.520760 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:08.520782 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:08.522539 (+ 1757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:08.524236 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:08.524241 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:08.524254 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:08.524491 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:08.524769 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:08.524772 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:08.524774 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:08.524905 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:08.525282 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:08.525289 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:08.525507 (+ 218us) client_negotiation.cc:769] Sending connection context
0808 06:05:08.525669 (+ 162us) client_negotiation.cc:240] Negotiation successful
0808 06:05:08.525985 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":141,"threads_started":1}
I20260808 06:05:08.528313 13853 heartbeater.cc:347] Connected to a master server at 127.1.151.190:41443
I20260808 06:05:08.528461 13853 heartbeater.cc:464] Registering TS with master...
I20260808 06:05:08.529007 13853 heartbeater.cc:511] Master 127.1.151.190:41443 requested a full tablet report, sending...
I20260808 06:05:08.530623 13536 ts_manager.cc:194] Registered new tserver with Master: d784a288c65342a8a9b126e4ec8a618f (127.1.151.129:35179)
I20260808 06:05:08.531713 13536 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.151.129@KRBTEST.COM'} at 127.1.151.129:43585
WARNING: no policy specified for oryx/127.1.151.130@KRBTEST.COM; defaulting to no policy
I20260808 06:05:08.539037 13865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:08.530259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:41443 (local address 127.1.151.129:53745)
0808 06:05:08.532345 (+ 2086us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:08.532359 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:08.532458 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:08.532732 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:08.532735 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:08.533019 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:08.533134 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:08.533152 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:08.534145 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:08.534171 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:08.534752 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:08.534762 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:08.535281 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:08.535284 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:08.535346 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:08.535802 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:08.535813 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:08.536093 (+ 280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:08.537729 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:08.537732 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:08.537734 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:08.537926 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:08.538201 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:08.538205 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:08.538207 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:08.538309 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:08.538643 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:08.538645 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:08.538729 (+ 84us) client_negotiation.cc:769] Sending connection context
0808 06:05:08.538805 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 06:05:08.538907 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2001,"thread_start_us":95,"threads_started":1}
I20260808 06:05:08.539069 13610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:08.530317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:53745 (local address 127.1.151.190:41443)
0808 06:05:08.530417 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:08.530421 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:08.532429 (+ 2008us) server_negotiation.cc:407] Connection header received
0808 06:05:08.532548 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:08.532551 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:08.532603 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:08.532690 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:08.533268 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:08.534014 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:08.534865 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:08.535186 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:08.535287 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:08.536233 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:08.536237 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:08.536239 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:08.536266 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:08.537611 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:08.538016 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:08.538019 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:08.538020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:08.538092 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:08.538403 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:08.538405 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:08.538406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:08.538562 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:08.538636 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:08.538877 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 06:05:08.538964 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.151.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.151.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.130.keytab.
Entry for principal oryx/127.1.151.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.130.keytab.
I20260808 06:05:08.554858 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.130:0
--local_ip_for_outbound_sockets=127.1.151.130
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:41443
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.130.keytab
--principal=oryx/127.1.151.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:44679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:05:08.663933 13868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:08.664161 13868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:08.664208 13868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:08.668120 13868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:08.668171 13868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:08.668213 13868 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 06:05:08.668264 13868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.130
I20260808 06:05:08.673225 13868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:44679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.130.keytab
--principal=oryx/127.1.151.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.151.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.151.190:41443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.130
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:05:08.674116 13868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:08.675082 13868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:08.684454 13878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:08.684581 13868 server_base.cc:1034] running on GCE node
W20260808 06:05:08.684522 13885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:08.684540 13879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:08.685070 13868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:08.685735 13868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:08.686906 13868 hybrid_clock.cc:648] HybridClock initialized: now 1786169108686883 us; error 34 us; skew 500 ppm
Aug 08 06:05:08 dist-test-slave-zj8q krb5kdc[13476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169108, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:08.689433 13868 init.cc:377] Logged in from keytab as oryx/127.1.151.130@KRBTEST.COM (short username oryx)
I20260808 06:05:08.690342 13868 webserver.cc:460] Webserver started at http://127.1.151.130:46297/ using document root <none> and password file <none>
I20260808 06:05:08.690899 13868 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:08.690966 13868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:08.691169 13868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:05:08.693172 13868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-1/data/instance:
uuid: "0ecfb7f3474e44a6a9039eadb187ea9e"
format_stamp: "Formatted at 2026-08-08 06:05:08 on dist-test-slave-zj8q"
server_key: "de130a8a9c0e571b850507a3f1f74589"
server_key_iv: "a425c64e85425c9d44237d26a7e8cabf"
server_key_version: "encryptionkey@0"
I20260808 06:05:08.693719 13868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance:
uuid: "0ecfb7f3474e44a6a9039eadb187ea9e"
format_stamp: "Formatted at 2026-08-08 06:05:08 on dist-test-slave-zj8q"
server_key: "de130a8a9c0e571b850507a3f1f74589"
server_key_iv: "a425c64e85425c9d44237d26a7e8cabf"
server_key_version: "encryptionkey@0"
I20260808 06:05:08.697326 13868 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 06:05:08.699746 13895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:08.701004 13868 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 06:05:08.701113 13868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-1/wal
uuid: "0ecfb7f3474e44a6a9039eadb187ea9e"
format_stamp: "Formatted at 2026-08-08 06:05:08 on dist-test-slave-zj8q"
server_key: "de130a8a9c0e571b850507a3f1f74589"
server_key_iv: "a425c64e85425c9d44237d26a7e8cabf"
server_key_version: "encryptionkey@0"
I20260808 06:05:08.701210 13868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:08.715034 13868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:08.717562 13868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:05:08.717722 13868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:08.719126 13868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:05:08.719177 13868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:08.719226 13868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:05:08.719252 13868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 06:05:08 dist-test-slave-zj8q krb5kdc[13476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169108, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM
I20260808 06:05:08.729321 13868 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.130:44373
I20260808 06:05:08.729482 14086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.130:44373 every 8 connection(s)
I20260808 06:05:08.730350 13868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
I20260808 06:05:08.731287 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 13868
I20260808 06:05:08.731408 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance
I20260808 06:05:08.731680 1630 external_mini_cluster.cc:1428] Setting key f43920a0b6247d31af2f2d89dbdd6fa3
I20260808 06:05:08.732416 13610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:08.719761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:57637 (local address 127.1.151.190:41443)
0808 06:05:08.719906 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:08.719911 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:08.720739 (+ 828us) server_negotiation.cc:407] Connection header received
0808 06:05:08.721768 (+ 1029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:08.721771 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:08.721832 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:08.721891 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:08.723201 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:08.723855 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:08.725059 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:08.725284 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:08.725373 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:08.728699 (+ 3326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:08.728702 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:08.728704 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:08.728729 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:08.730177 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:08.730706 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:08.730709 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:08.730710 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:08.730807 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:08.731335 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:08.731338 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:08.731340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:08.731529 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:08.731619 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:08.732136 (+ 517us) server_negotiation.cc:299] Negotiation successful
0808 06:05:08.732253 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 06:05:08.733238 13908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:08.720114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:41443 (local address 127.1.151.130:57637)
0808 06:05:08.720568 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:08.720617 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:08.721504 (+ 887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:08.722268 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:08.722275 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:08.722576 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:08.723052 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:08.723066 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:08.723981 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:08.723985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:08.724937 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:08.724948 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:08.725438 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:08.725442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:08.725615 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:08.726141 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:08.726183 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:08.728535 (+ 2352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:08.730276 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:08.730282 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:08.730295 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:08.730609 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:08.731059 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:08.731062 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:08.731064 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:08.731223 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:08.731635 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:08.731641 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:08.731892 (+ 251us) client_negotiation.cc:769] Sending connection context
0808 06:05:08.732066 (+ 174us) client_negotiation.cc:240] Negotiation successful
0808 06:05:08.732377 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":98,"threads_started":1}
I20260808 06:05:08.734519 14087 heartbeater.cc:347] Connected to a master server at 127.1.151.190:41443
I20260808 06:05:08.734647 14087 heartbeater.cc:464] Registering TS with master...
I20260808 06:05:08.735204 14087 heartbeater.cc:511] Master 127.1.151.190:41443 requested a full tablet report, sending...
I20260808 06:05:08.736328 13536 ts_manager.cc:194] Registered new tserver with Master: 0ecfb7f3474e44a6a9039eadb187ea9e (127.1.151.130:44373)
I20260808 06:05:08.736954 13536 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.151.130@KRBTEST.COM'} at 127.1.151.130:57637
WARNING: no policy specified for oryx/127.1.151.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.151.131@KRBTEST.COM" created.
I20260808 06:05:08.744778 13610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:08.736559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:51627 (local address 127.1.151.190:41443)
0808 06:05:08.736670 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:08.736674 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:08.737391 (+ 717us) server_negotiation.cc:407] Connection header received
0808 06:05:08.738847 (+ 1456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:08.738850 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:08.738890 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:08.738954 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:08.739386 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:08.739869 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:08.740645 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:08.740849 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:08.740932 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:08.741975 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:08.741977 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:08.741979 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:08.742000 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:08.743243 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:08.743620 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:08.743623 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:08.743624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:08.743702 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:08.743981 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:08.743983 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:08.743984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:08.744190 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:08.744274 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:08.744562 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 06:05:08.744664 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 06:05:08.744814 14097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:08.736564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:41443 (local address 127.1.151.130:51627)
0808 06:05:08.737316 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:08.737330 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:08.737416 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:08.738984 (+ 1568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:08.738987 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:08.739178 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:08.739278 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:08.739283 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:08.739985 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:08.739988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:08.740531 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:08.740539 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:08.740946 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:08.740949 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:08.741043 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:08.741527 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:08.741537 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:08.741846 (+ 309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:08.743344 (+ 1498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:08.743347 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:08.743349 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:08.743532 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:08.743785 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:08.743788 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:08.743789 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:08.743905 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:08.744271 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:08.744274 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:08.744369 (+ 95us) client_negotiation.cc:769] Sending connection context
0808 06:05:08.744473 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 06:05:08.744618 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"thread_start_us":101,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.151.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.131.keytab.
Entry for principal oryx/127.1.151.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.131.keytab.
I20260808 06:05:08.759428 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.131:0
--local_ip_for_outbound_sockets=127.1.151.131
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:41443
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.131.keytab
--principal=oryx/127.1.151.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:44679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:05:08.868719 14105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:08.868949 14105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:08.868992 14105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:08.872874 14105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:08.872923 14105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:08.872963 14105 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 06:05:08.873014 14105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.131
I20260808 06:05:08.878466 14105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:44679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.131.keytab
--principal=oryx/127.1.151.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.151.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.151.190:41443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.131
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:05:08.879434 14105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:08.880370 14105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:08.889722 14113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:08.889791 14105 server_base.cc:1034] running on GCE node
W20260808 06:05:08.889752 14111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:08.889967 14116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:08.890360 14105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:08.890944 14105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:08.892107 14105 hybrid_clock.cc:648] HybridClock initialized: now 1786169108892087 us; error 30 us; skew 500 ppm
Aug 08 06:05:08 dist-test-slave-zj8q krb5kdc[13476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169108, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:08.894744 14105 init.cc:377] Logged in from keytab as oryx/127.1.151.131@KRBTEST.COM (short username oryx)
I20260808 06:05:08.895480 14105 webserver.cc:460] Webserver started at http://127.1.151.131:36707/ using document root <none> and password file <none>
I20260808 06:05:08.895982 14105 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:08.896040 14105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:08.896200 14105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:05:08.898056 14105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-2/data/instance:
uuid: "d2678bd25fe44d98b1515f2188a9d777"
format_stamp: "Formatted at 2026-08-08 06:05:08 on dist-test-slave-zj8q"
server_key: "4177c40868d5a6beb789bfc8f5089dfd"
server_key_iv: "c74176d925d87a2f99246e9ac13ec0f5"
server_key_version: "encryptionkey@0"
I20260808 06:05:08.898589 14105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance:
uuid: "d2678bd25fe44d98b1515f2188a9d777"
format_stamp: "Formatted at 2026-08-08 06:05:08 on dist-test-slave-zj8q"
server_key: "4177c40868d5a6beb789bfc8f5089dfd"
server_key_iv: "c74176d925d87a2f99246e9ac13ec0f5"
server_key_version: "encryptionkey@0"
I20260808 06:05:08.902138 14105 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 06:05:08.904548 14125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:08.905761 14105 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 06:05:08.905859 14105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-2/wal
uuid: "d2678bd25fe44d98b1515f2188a9d777"
format_stamp: "Formatted at 2026-08-08 06:05:08 on dist-test-slave-zj8q"
server_key: "4177c40868d5a6beb789bfc8f5089dfd"
server_key_iv: "c74176d925d87a2f99246e9ac13ec0f5"
server_key_version: "encryptionkey@0"
I20260808 06:05:08.905958 14105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:08.940313 14105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:08.943150 14105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:05:08.943326 14105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:08.944747 14105 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:05:08.944801 14105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:08.944855 14105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:05:08.944882 14105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 06:05:08 dist-test-slave-zj8q krb5kdc[13476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169108, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM
I20260808 06:05:08.954593 14105 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.131:35921
I20260808 06:05:08.954649 14298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.131:35921 every 8 connection(s)
I20260808 06:05:08.955580 14105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
I20260808 06:05:08.957796 13610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:08.945588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:50731 (local address 127.1.151.190:41443)
0808 06:05:08.945743 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:08.945747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:08.946811 (+ 1064us) server_negotiation.cc:407] Connection header received
0808 06:05:08.947848 (+ 1037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:08.947852 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:08.947923 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:08.947990 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:08.949617 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:08.950473 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:08.951603 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:08.951916 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:08.952014 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:08.954564 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:08.954568 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:08.954571 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:08.954611 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:08.955927 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:08.956500 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:08.956504 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:08.956506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:08.956604 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:08.956869 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:08.956872 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:08.956873 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:08.957042 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:08.957122 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:08.957580 (+ 458us) server_negotiation.cc:299] Negotiation successful
0808 06:05:08.957677 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 06:05:08.958582 14144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:08.946138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:41443 (local address 127.1.151.131:50731)
0808 06:05:08.946663 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:08.946705 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:08.947585 (+ 880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:08.948408 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:08.948418 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:08.948809 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:08.949447 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:08.949466 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:08.950627 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:08.950630 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:08.951472 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:08.951481 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:08.952062 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:08.952066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:08.952244 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:08.952882 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:08.952897 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:08.954404 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:08.956065 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:08.956071 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:08.956083 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:08.956400 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:08.956690 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:08.956691 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:08.956693 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:08.956809 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:08.957163 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:08.957169 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:08.957388 (+ 219us) client_negotiation.cc:769] Sending connection context
0808 06:05:08.957551 (+ 163us) client_negotiation.cc:240] Negotiation successful
0808 06:05:08.957790 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":134,"threads_started":1}
I20260808 06:05:08.959674 14301 heartbeater.cc:347] Connected to a master server at 127.1.151.190:41443
I20260808 06:05:08.959805 14301 heartbeater.cc:464] Registering TS with master...
I20260808 06:05:08.960248 14301 heartbeater.cc:511] Master 127.1.151.190:41443 requested a full tablet report, sending...
I20260808 06:05:08.961171 13536 ts_manager.cc:194] Registered new tserver with Master: d2678bd25fe44d98b1515f2188a9d777 (127.1.151.131:35921)
I20260808 06:05:08.961774 13536 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.151.131@KRBTEST.COM'} at 127.1.151.131:50731
I20260808 06:05:08.965672 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 14105
I20260808 06:05:08.965775 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance
I20260808 06:05:08.966037 1630 external_mini_cluster.cc:1428] Setting key 6b5dee2242ff8c949da395e2df22b7d7
I20260808 06:05:08.968092 1630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 06:05:08.968262 14310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:08.961558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:41443 (local address 127.1.151.131:33759)
0808 06:05:08.961837 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:08.961852 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:08.961943 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:08.962191 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:08.962194 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:08.962390 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:08.962483 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:08.962488 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:08.963467 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:08.963470 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:08.964026 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:08.964034 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:08.964423 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:08.964425 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:08.964491 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:08.964864 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:08.964874 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:08.965142 (+ 268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:08.966806 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:08.966810 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:08.966812 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:08.967039 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:08.967338 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:08.967341 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:08.967342 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:08.967417 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:08.967778 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:08.967780 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:08.967864 (+ 84us) client_negotiation.cc:769] Sending connection context
0808 06:05:08.967963 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 06:05:08.968081 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":97,"threads_started":1}
I20260808 06:05:08.968283 13610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:08.961660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:33759 (local address 127.1.151.190:41443)
0808 06:05:08.961775 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:08.961779 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:08.961910 (+ 131us) server_negotiation.cc:407] Connection header received
0808 06:05:08.962023 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:08.962026 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:08.962073 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:08.962181 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:08.962591 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:08.963314 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:08.964130 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:08.964338 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:08.964415 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:08.965252 (+ 837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:08.965255 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:08.965257 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:08.965277 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:08.966688 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:08.967158 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:08.967161 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:08.967163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:08.967250 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:08.967494 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:08.967497 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:08.967499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:08.967695 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:08.967785 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:08.968032 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 06:05:08.968133 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Aug 08 06:05:08 dist-test-slave-zj8q krb5kdc[13476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169108, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 06:05:08 dist-test-slave-zj8q krb5kdc[13476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:08 dist-test-slave-zj8q krb5kdc[13476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 06:05:08.991617 13610 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:08.985868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47728 (local address 127.1.151.190:41443)
0808 06:05:08.985996 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:08.986000 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:08.986011 (+ 11us) server_negotiation.cc:407] Connection header received
0808 06:05:08.986043 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:08.986045 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:08.986078 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:08.986176 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:08.986854 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:08.987351 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:08.988187 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:08.988391 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:08.988477 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:08.991470 (+ 2993us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47728: BlockingRecv error: recv got EOF from 127.0.0.1:47728 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 06:05:08 dist-test-slave-zj8q krb5kdc[13476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169108, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM
I20260808 06:05:09.000097 13610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:08.992904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47736 (local address 127.1.151.190:41443)
0808 06:05:08.993010 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:08.993014 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:08.993103 (+ 89us) server_negotiation.cc:407] Connection header received
0808 06:05:08.993197 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:08.993201 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:08.993238 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:08.993299 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:08.993858 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:08.994429 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:08.995209 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:08.995418 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:08.995499 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:08.997443 (+ 1944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:08.997446 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:08.997448 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:08.997469 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:08.998650 (+ 1181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:08.999015 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:08.999019 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:08.999020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:08.999099 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:08.999343 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:08.999346 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:08.999348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:08.999503 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:08.999672 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:08.999855 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 06:05:08.999955 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 06:05:09.002772 13536 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47736:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 06:05:09.005955 13536 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 06:05:09.019577 14344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:09.014976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47214 (local address 127.1.151.131:35921)
0808 06:05:09.015396 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:09.015409 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:09.015430 (+ 21us) server_negotiation.cc:407] Connection header received
0808 06:05:09.015502 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:09.015507 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:09.015703 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:09.015849 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:05:09.016063 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.016912 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.018269 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.018871 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.018983 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:09.019159 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:09.019328 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 06:05:09.019429 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":117,"threads_started":1}
I20260808 06:05:09.019562 14338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:09.014029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:35921 (local address 127.0.0.1:47214)
0808 06:05:09.014563 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:09.014579 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:09.014678 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:09.015841 (+ 1163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:09.015844 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:09.015861 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:05:09.015947 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:09.015952 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.017027 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.017030 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:09.018095 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:09.018107 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.018996 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.019000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:09.019111 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:09.019177 (+ 66us) client_negotiation.cc:769] Sending connection context
0808 06:05:09.019288 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 06:05:09.019411 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"mutex_wait_us":5,"thread_start_us":135,"threads_started":1}
I20260808 06:05:09.019724 14339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:09.013914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:44373 (local address 127.0.0.1:38722)
0808 06:05:09.014415 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:09.014460 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:09.014583 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:09.015366 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:09.015372 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:09.015396 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:05:09.015525 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:09.015531 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.016848 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.016852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:09.018037 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:09.018048 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.019076 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.019079 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:09.019419 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:09.019440 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 06:05:09.019504 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 06:05:09.019563 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":236,"thread_start_us":68,"threads_started":1}
I20260808 06:05:09.020218 14341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:09.014593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:35179 (local address 127.0.0.1:41262)
0808 06:05:09.015380 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:09.015392 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:09.015529 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:09.016130 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:09.016133 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:09.016151 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:05:09.016240 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:09.016247 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.017507 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.017511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:09.018591 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:09.018613 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.019848 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.019852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:09.019941 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:09.019964 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 06:05:09.020043 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 06:05:09.020115 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":729,"thread_start_us":86,"threads_started":1}
I20260808 06:05:09.020293 14340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:09.014091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38722 (local address 127.1.151.130:44373)
0808 06:05:09.014828 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:09.014841 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:09.014864 (+ 23us) server_negotiation.cc:407] Connection header received
0808 06:05:09.014945 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:09.014951 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:09.015182 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:09.015674 (+ 492us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:05:09.015793 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.016728 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.018226 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.018948 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.019194 (+ 246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:09.019350 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:09.020042 (+ 692us) server_negotiation.cc:299] Negotiation successful
0808 06:05:09.020156 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":142,"threads_started":1}
I20260808 06:05:09.022648 14206 tablet_service.cc:1467] Processing CreateTablet for tablet 8df38518f10b4caeaf1dde196e89305d (DEFAULT_TABLE table=test-table [id=255e1a8daf0e496eb5bec0069aa7c49d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:05:09.022646 13973 tablet_service.cc:1467] Processing CreateTablet for tablet 8df38518f10b4caeaf1dde196e89305d (DEFAULT_TABLE table=test-table [id=255e1a8daf0e496eb5bec0069aa7c49d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:05:09.023552 13973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8df38518f10b4caeaf1dde196e89305d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:05:09.023552 14206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8df38518f10b4caeaf1dde196e89305d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:05:09.024499 14342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:09.014862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41262 (local address 127.1.151.129:35179)
0808 06:05:09.015225 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:09.015242 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:09.015465 (+ 223us) server_negotiation.cc:407] Connection header received
0808 06:05:09.015677 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:09.015684 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:09.015851 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:09.015978 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:05:09.016369 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.017379 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.018732 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.019497 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.019604 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:09.019784 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:09.020473 (+ 689us) server_negotiation.cc:299] Negotiation successful
0808 06:05:09.020574 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":80,"threads_started":1}
I20260808 06:05:09.027273 13750 tablet_service.cc:1467] Processing CreateTablet for tablet 8df38518f10b4caeaf1dde196e89305d (DEFAULT_TABLE table=test-table [id=255e1a8daf0e496eb5bec0069aa7c49d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:05:09.028347 13750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8df38518f10b4caeaf1dde196e89305d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:05:09.030992 14350 tablet_bootstrap.cc:492] T 8df38518f10b4caeaf1dde196e89305d P d2678bd25fe44d98b1515f2188a9d777: Bootstrap starting.
I20260808 06:05:09.031932 14347 tablet_bootstrap.cc:492] T 8df38518f10b4caeaf1dde196e89305d P 0ecfb7f3474e44a6a9039eadb187ea9e: Bootstrap starting.
I20260808 06:05:09.032996 14350 tablet_bootstrap.cc:654] T 8df38518f10b4caeaf1dde196e89305d P d2678bd25fe44d98b1515f2188a9d777: Neither blocks nor log segments found. Creating new log.
I20260808 06:05:09.033735 14350 log.cc:824] T 8df38518f10b4caeaf1dde196e89305d P d2678bd25fe44d98b1515f2188a9d777: Log is configured to *not* fsync() on all Append() calls
I20260808 06:05:09.034034 14347 tablet_bootstrap.cc:654] T 8df38518f10b4caeaf1dde196e89305d P 0ecfb7f3474e44a6a9039eadb187ea9e: Neither blocks nor log segments found. Creating new log.
I20260808 06:05:09.034864 14347 log.cc:824] T 8df38518f10b4caeaf1dde196e89305d P 0ecfb7f3474e44a6a9039eadb187ea9e: Log is configured to *not* fsync() on all Append() calls
I20260808 06:05:09.035368 14352 tablet_bootstrap.cc:492] T 8df38518f10b4caeaf1dde196e89305d P d784a288c65342a8a9b126e4ec8a618f: Bootstrap starting.
I20260808 06:05:09.036404 14350 tablet_bootstrap.cc:492] T 8df38518f10b4caeaf1dde196e89305d P d2678bd25fe44d98b1515f2188a9d777: No bootstrap required, opened a new log
I20260808 06:05:09.036716 14350 ts_tablet_manager.cc:1397] T 8df38518f10b4caeaf1dde196e89305d P d2678bd25fe44d98b1515f2188a9d777: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 06:05:09.036782 14347 tablet_bootstrap.cc:492] T 8df38518f10b4caeaf1dde196e89305d P 0ecfb7f3474e44a6a9039eadb187ea9e: No bootstrap required, opened a new log
I20260808 06:05:09.036950 14347 ts_tablet_manager.cc:1397] T 8df38518f10b4caeaf1dde196e89305d P 0ecfb7f3474e44a6a9039eadb187ea9e: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260808 06:05:09.037894 14352 tablet_bootstrap.cc:654] T 8df38518f10b4caeaf1dde196e89305d P d784a288c65342a8a9b126e4ec8a618f: Neither blocks nor log segments found. Creating new log.
I20260808 06:05:09.038725 14352 log.cc:824] T 8df38518f10b4caeaf1dde196e89305d P d784a288c65342a8a9b126e4ec8a618f: Log is configured to *not* fsync() on all Append() calls
I20260808 06:05:09.039922 14347 raft_consensus.cc:348] T 8df38518f10b4caeaf1dde196e89305d P 0ecfb7f3474e44a6a9039eadb187ea9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } } peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } }
I20260808 06:05:09.040127 14347 raft_consensus.cc:374] T 8df38518f10b4caeaf1dde196e89305d P 0ecfb7f3474e44a6a9039eadb187ea9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:05:09.040198 14347 raft_consensus.cc:729] T 8df38518f10b4caeaf1dde196e89305d P 0ecfb7f3474e44a6a9039eadb187ea9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ecfb7f3474e44a6a9039eadb187ea9e, State: Initialized, Role: FOLLOWER
I20260808 06:05:09.040624 14352 tablet_bootstrap.cc:492] T 8df38518f10b4caeaf1dde196e89305d P d784a288c65342a8a9b126e4ec8a618f: No bootstrap required, opened a new log
I20260808 06:05:09.040581 14347 consensus_queue.cc:260] T 8df38518f10b4caeaf1dde196e89305d P 0ecfb7f3474e44a6a9039eadb187ea9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } } peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } }
I20260808 06:05:09.040495 14350 raft_consensus.cc:348] T 8df38518f10b4caeaf1dde196e89305d P d2678bd25fe44d98b1515f2188a9d777 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } } peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } }
I20260808 06:05:09.040782 14350 raft_consensus.cc:374] T 8df38518f10b4caeaf1dde196e89305d P d2678bd25fe44d98b1515f2188a9d777 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:05:09.040815 14352 ts_tablet_manager.cc:1397] T 8df38518f10b4caeaf1dde196e89305d P d784a288c65342a8a9b126e4ec8a618f: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260808 06:05:09.040846 14350 raft_consensus.cc:729] T 8df38518f10b4caeaf1dde196e89305d P d2678bd25fe44d98b1515f2188a9d777 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2678bd25fe44d98b1515f2188a9d777, State: Initialized, Role: FOLLOWER
I20260808 06:05:09.041292 14350 consensus_queue.cc:260] T 8df38518f10b4caeaf1dde196e89305d P d2678bd25fe44d98b1515f2188a9d777 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } } peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } }
I20260808 06:05:09.041424 14347 ts_tablet_manager.cc:1428] T 8df38518f10b4caeaf1dde196e89305d P 0ecfb7f3474e44a6a9039eadb187ea9e: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 06:05:09.041821 14087 heartbeater.cc:503] Master 127.1.151.190:41443 was elected leader, sending a full tablet report...
I20260808 06:05:09.042279 14350 ts_tablet_manager.cc:1428] T 8df38518f10b4caeaf1dde196e89305d P d2678bd25fe44d98b1515f2188a9d777: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 06:05:09.042519 14301 heartbeater.cc:503] Master 127.1.151.190:41443 was elected leader, sending a full tablet report...
I20260808 06:05:09.045305 14352 raft_consensus.cc:348] T 8df38518f10b4caeaf1dde196e89305d P d784a288c65342a8a9b126e4ec8a618f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } } peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } }
I20260808 06:05:09.045567 14352 raft_consensus.cc:374] T 8df38518f10b4caeaf1dde196e89305d P d784a288c65342a8a9b126e4ec8a618f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:05:09.045631 14352 raft_consensus.cc:729] T 8df38518f10b4caeaf1dde196e89305d P d784a288c65342a8a9b126e4ec8a618f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d784a288c65342a8a9b126e4ec8a618f, State: Initialized, Role: FOLLOWER
I20260808 06:05:09.046013 14352 consensus_queue.cc:260] T 8df38518f10b4caeaf1dde196e89305d P d784a288c65342a8a9b126e4ec8a618f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } } peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } }
I20260808 06:05:09.046587 13853 heartbeater.cc:503] Master 127.1.151.190:41443 was elected leader, sending a full tablet report...
I20260808 06:05:09.046941 14352 ts_tablet_manager.cc:1428] T 8df38518f10b4caeaf1dde196e89305d P d784a288c65342a8a9b126e4ec8a618f: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
W20260808 06:05:09.206990 14302 tablet.cc:2369] T 8df38518f10b4caeaf1dde196e89305d P d2678bd25fe44d98b1515f2188a9d777: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 06:05:09.209393 14356 raft_consensus.cc:482] T 8df38518f10b4caeaf1dde196e89305d P 0ecfb7f3474e44a6a9039eadb187ea9e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:05:09.209586 14356 raft_consensus.cc:504] T 8df38518f10b4caeaf1dde196e89305d P 0ecfb7f3474e44a6a9039eadb187ea9e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } } peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } }
I20260808 06:05:09.210806 14356 leader_election.cc:288] T 8df38518f10b4caeaf1dde196e89305d P 0ecfb7f3474e44a6a9039eadb187ea9e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2678bd25fe44d98b1515f2188a9d777 (127.1.151.131:35921), d784a288c65342a8a9b126e4ec8a618f (127.1.151.129:35179)
I20260808 06:05:09.215132 13908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:09.211116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:35921 (local address 127.1.151.130:50399)
0808 06:05:09.211249 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:09.211265 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:09.211365 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:09.211708 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:09.211712 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:09.211731 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:05:09.211871 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:09.211878 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.212912 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.212915 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:09.214119 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:09.214132 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.214791 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.214793 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:09.214890 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:09.214911 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 06:05:09.214969 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 06:05:09.215026 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 06:05:09.215189 14344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:09.211225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:50399 (local address 127.1.151.131:35921)
0808 06:05:09.211356 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:09.211361 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:09.211381 (+ 20us) server_negotiation.cc:407] Connection header received
0808 06:05:09.211449 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:09.211452 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:09.211499 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:09.211585 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:05:09.211983 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.212792 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.214268 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.214677 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.214794 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:09.214847 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:09.214992 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 06:05:09.215063 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 06:05:09.215752 14236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8df38518f10b4caeaf1dde196e89305d" candidate_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2678bd25fe44d98b1515f2188a9d777" is_pre_election: true
I20260808 06:05:09.215863 14359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:09.211412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:35179 (local address 127.1.151.130:57903)
0808 06:05:09.211720 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:09.211732 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:09.211829 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:09.212056 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:09.212058 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:09.212074 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:05:09.212180 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:09.212184 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.213085 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.213088 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:09.214599 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:09.214608 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.215444 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.215448 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:09.215553 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:09.215573 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 06:05:09.215639 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 06:05:09.215705 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":1,"thread_start_us":106,"threads_started":1}
I20260808 06:05:09.215854 14342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:09.211535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:57903 (local address 127.1.151.129:35179)
0808 06:05:09.211656 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:09.211660 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:09.211809 (+ 149us) server_negotiation.cc:407] Connection header received
0808 06:05:09.211904 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:09.211906 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:09.211950 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:09.212022 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:05:09.212271 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.212978 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.214721 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.215250 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.215361 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:09.215407 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:09.215677 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 06:05:09.215750 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 06:05:09.216058 14236 raft_consensus.cc:2454] T 8df38518f10b4caeaf1dde196e89305d P d2678bd25fe44d98b1515f2188a9d777 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ecfb7f3474e44a6a9039eadb187ea9e in term 0.
I20260808 06:05:09.216394 13779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8df38518f10b4caeaf1dde196e89305d" candidate_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d784a288c65342a8a9b126e4ec8a618f" is_pre_election: true
I20260808 06:05:09.216558 13897 leader_election.cc:302] T 8df38518f10b4caeaf1dde196e89305d P 0ecfb7f3474e44a6a9039eadb187ea9e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ecfb7f3474e44a6a9039eadb187ea9e, d2678bd25fe44d98b1515f2188a9d777; no voters:
I20260808 06:05:09.216653 13779 raft_consensus.cc:2454] T 8df38518f10b4caeaf1dde196e89305d P d784a288c65342a8a9b126e4ec8a618f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ecfb7f3474e44a6a9039eadb187ea9e in term 0.
I20260808 06:05:09.216820 14356 raft_consensus.cc:2781] T 8df38518f10b4caeaf1dde196e89305d P 0ecfb7f3474e44a6a9039eadb187ea9e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 06:05:09.216918 14356 raft_consensus.cc:482] T 8df38518f10b4caeaf1dde196e89305d P 0ecfb7f3474e44a6a9039eadb187ea9e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 06:05:09.216971 14356 raft_consensus.cc:3037] T 8df38518f10b4caeaf1dde196e89305d P 0ecfb7f3474e44a6a9039eadb187ea9e [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:05:09.217989 14356 raft_consensus.cc:504] T 8df38518f10b4caeaf1dde196e89305d P 0ecfb7f3474e44a6a9039eadb187ea9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } } peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } }
I20260808 06:05:09.218360 14356 leader_election.cc:288] T 8df38518f10b4caeaf1dde196e89305d P 0ecfb7f3474e44a6a9039eadb187ea9e [CANDIDATE]: Term 1 election: Requested vote from peers d2678bd25fe44d98b1515f2188a9d777 (127.1.151.131:35921), d784a288c65342a8a9b126e4ec8a618f (127.1.151.129:35179)
I20260808 06:05:09.218720 14236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8df38518f10b4caeaf1dde196e89305d" candidate_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2678bd25fe44d98b1515f2188a9d777"
I20260808 06:05:09.218745 13779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8df38518f10b4caeaf1dde196e89305d" candidate_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d784a288c65342a8a9b126e4ec8a618f"
I20260808 06:05:09.218832 14236 raft_consensus.cc:3037] T 8df38518f10b4caeaf1dde196e89305d P d2678bd25fe44d98b1515f2188a9d777 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:05:09.218840 13779 raft_consensus.cc:3037] T 8df38518f10b4caeaf1dde196e89305d P d784a288c65342a8a9b126e4ec8a618f [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:05:09.219838 13779 raft_consensus.cc:2454] T 8df38518f10b4caeaf1dde196e89305d P d784a288c65342a8a9b126e4ec8a618f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ecfb7f3474e44a6a9039eadb187ea9e in term 1.
I20260808 06:05:09.220104 13897 leader_election.cc:302] T 8df38518f10b4caeaf1dde196e89305d P 0ecfb7f3474e44a6a9039eadb187ea9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ecfb7f3474e44a6a9039eadb187ea9e, d784a288c65342a8a9b126e4ec8a618f; no voters:
I20260808 06:05:09.220270 14236 raft_consensus.cc:2454] T 8df38518f10b4caeaf1dde196e89305d P d2678bd25fe44d98b1515f2188a9d777 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ecfb7f3474e44a6a9039eadb187ea9e in term 1.
I20260808 06:05:09.220299 14356 raft_consensus.cc:2781] T 8df38518f10b4caeaf1dde196e89305d P 0ecfb7f3474e44a6a9039eadb187ea9e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:05:09.220476 14356 raft_consensus.cc:686] T 8df38518f10b4caeaf1dde196e89305d P 0ecfb7f3474e44a6a9039eadb187ea9e [term 1 LEADER]: Becoming Leader. State: Replica: 0ecfb7f3474e44a6a9039eadb187ea9e, State: Running, Role: LEADER
I20260808 06:05:09.220791 14356 consensus_queue.cc:237] T 8df38518f10b4caeaf1dde196e89305d P 0ecfb7f3474e44a6a9039eadb187ea9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } } peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } }
I20260808 06:05:09.223902 13536 catalog_manager.cc:5495] T 8df38518f10b4caeaf1dde196e89305d P 0ecfb7f3474e44a6a9039eadb187ea9e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ecfb7f3474e44a6a9039eadb187ea9e (127.1.151.130). New cstate: current_term: 1 leader_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } health_report { overall_health: UNKNOWN } } }
W20260808 06:05:09.232475 14088 tablet.cc:2369] T 8df38518f10b4caeaf1dde196e89305d P 0ecfb7f3474e44a6a9039eadb187ea9e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 06:05:09.254233 14340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:09.250874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38728 (local address 127.1.151.130:44373)
0808 06:05:09.250990 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:09.250994 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:09.251006 (+ 12us) server_negotiation.cc:407] Connection header received
0808 06:05:09.251123 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:09.251126 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:09.251184 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:09.251269 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:05:09.251470 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.251961 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.252802 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.253024 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.253141 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:09.253405 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:05:09.253818 (+ 413us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:05:09.253901 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:09.254007 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 06:05:09.254056 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 06:05:09.259764 13779 raft_consensus.cc:1261] T 8df38518f10b4caeaf1dde196e89305d P d784a288c65342a8a9b126e4ec8a618f [term 1 FOLLOWER]: Refusing update from remote peer 0ecfb7f3474e44a6a9039eadb187ea9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:05:09.259773 14236 raft_consensus.cc:1261] T 8df38518f10b4caeaf1dde196e89305d P d2678bd25fe44d98b1515f2188a9d777 [term 1 FOLLOWER]: Refusing update from remote peer 0ecfb7f3474e44a6a9039eadb187ea9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:05:09.260499 14360 consensus_queue.cc:1035] T 8df38518f10b4caeaf1dde196e89305d P 0ecfb7f3474e44a6a9039eadb187ea9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:05:09.260753 14356 consensus_queue.cc:1035] T 8df38518f10b4caeaf1dde196e89305d P 0ecfb7f3474e44a6a9039eadb187ea9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:05:09.269887 14366 mvcc.cc:205] Tried to move back new op lower bound from 7316148671520296960 to 7316148671370678272. Current Snapshot: MvccSnapshot[applied={T|T < 7316148671520296960}]
I20260808 06:05:09.270412 14364 mvcc.cc:205] Tried to move back new op lower bound from 7316148671520296960 to 7316148671370678272. Current Snapshot: MvccSnapshot[applied={T|T < 7316148671520296960 or (T in {7316148671520296960})}]
I20260808 06:05:09.277104 13535 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:05:09.277278 13535 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:05:09.279676 13535 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f5f0153a8bbb4a819b1bb2eebff8ca4f: Sending DeleteTablet for 3 replicas of tablet 8df38518f10b4caeaf1dde196e89305d
I20260808 06:05:09.280563 13973 tablet_service.cc:1514] Processing DeleteTablet for tablet 8df38518f10b4caeaf1dde196e89305d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:05:09 UTC) from {username='oryx', principal='oryx/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:38722
I20260808 06:05:09.280563 13750 tablet_service.cc:1514] Processing DeleteTablet for tablet 8df38518f10b4caeaf1dde196e89305d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:05:09 UTC) from {username='oryx', principal='oryx/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:41262
I20260808 06:05:09.280998 14206 tablet_service.cc:1514] Processing DeleteTablet for tablet 8df38518f10b4caeaf1dde196e89305d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:05:09 UTC) from {username='oryx', principal='oryx/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:47214
I20260808 06:05:09.280943 13535 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47736:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 06:05:09.281334 13535 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 06:05:09.281402 14386 tablet_replica.cc:330] T 8df38518f10b4caeaf1dde196e89305d P d2678bd25fe44d98b1515f2188a9d777: stopping tablet replica
I20260808 06:05:09.281673 14384 tablet_replica.cc:330] T 8df38518f10b4caeaf1dde196e89305d P d784a288c65342a8a9b126e4ec8a618f: stopping tablet replica
I20260808 06:05:09.281705 14386 raft_consensus.cc:2229] T 8df38518f10b4caeaf1dde196e89305d P d2678bd25fe44d98b1515f2188a9d777 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:05:09.281997 14384 raft_consensus.cc:2229] T 8df38518f10b4caeaf1dde196e89305d P d784a288c65342a8a9b126e4ec8a618f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:05:09.281996 14386 raft_consensus.cc:2258] T 8df38518f10b4caeaf1dde196e89305d P d2678bd25fe44d98b1515f2188a9d777 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:05:09.282336 14384 raft_consensus.cc:2258] T 8df38518f10b4caeaf1dde196e89305d P d784a288c65342a8a9b126e4ec8a618f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:05:09.283571 14386 ts_tablet_manager.cc:1905] T 8df38518f10b4caeaf1dde196e89305d P d2678bd25fe44d98b1515f2188a9d777: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:05:09.285737 14385 tablet_replica.cc:330] T 8df38518f10b4caeaf1dde196e89305d P 0ecfb7f3474e44a6a9039eadb187ea9e: stopping tablet replica
I20260808 06:05:09.286012 14385 raft_consensus.cc:2229] T 8df38518f10b4caeaf1dde196e89305d P 0ecfb7f3474e44a6a9039eadb187ea9e [term 1 LEADER]: Raft consensus shutting down.
I20260808 06:05:09.286335 13750 tablet_service.cc:1467] Processing CreateTablet for tablet 73fa8a6c21d04ae98b2a7f0b856c57d8 (DEFAULT_TABLE table=test-table [id=b9cdcf0ded28487e99b4af14219983d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:05:09.286374 14386 ts_tablet_manager.cc:1918] T 8df38518f10b4caeaf1dde196e89305d P d2678bd25fe44d98b1515f2188a9d777: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:05:09.286466 14386 log.cc:1196] T 8df38518f10b4caeaf1dde196e89305d P d2678bd25fe44d98b1515f2188a9d777: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-2/wal/wals/8df38518f10b4caeaf1dde196e89305d
I20260808 06:05:09.286450 14385 raft_consensus.cc:2258] T 8df38518f10b4caeaf1dde196e89305d P 0ecfb7f3474e44a6a9039eadb187ea9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:05:09.286598 13750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73fa8a6c21d04ae98b2a7f0b856c57d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:05:09.286589 14206 tablet_service.cc:1467] Processing CreateTablet for tablet 73fa8a6c21d04ae98b2a7f0b856c57d8 (DEFAULT_TABLE table=test-table [id=b9cdcf0ded28487e99b4af14219983d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:05:09.286791 14386 ts_tablet_manager.cc:1939] T 8df38518f10b4caeaf1dde196e89305d P d2678bd25fe44d98b1515f2188a9d777: Deleting consensus metadata
I20260808 06:05:09.286832 14206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73fa8a6c21d04ae98b2a7f0b856c57d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:05:09.287952 13518 catalog_manager.cc:4841] TS d2678bd25fe44d98b1515f2188a9d777 (127.1.151.131:35921): tablet 8df38518f10b4caeaf1dde196e89305d (table test-table [id=255e1a8daf0e496eb5bec0069aa7c49d]) successfully deleted
I20260808 06:05:09.289613 14350 tablet_bootstrap.cc:492] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d2678bd25fe44d98b1515f2188a9d777: Bootstrap starting.
I20260808 06:05:09.289978 14384 ts_tablet_manager.cc:1905] T 8df38518f10b4caeaf1dde196e89305d P d784a288c65342a8a9b126e4ec8a618f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:05:09.290729 14350 tablet_bootstrap.cc:654] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d2678bd25fe44d98b1515f2188a9d777: Neither blocks nor log segments found. Creating new log.
I20260808 06:05:09.291095 14352 tablet_bootstrap.cc:492] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d784a288c65342a8a9b126e4ec8a618f: Bootstrap starting.
I20260808 06:05:09.292029 14352 tablet_bootstrap.cc:654] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d784a288c65342a8a9b126e4ec8a618f: Neither blocks nor log segments found. Creating new log.
I20260808 06:05:09.292109 14350 tablet_bootstrap.cc:492] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d2678bd25fe44d98b1515f2188a9d777: No bootstrap required, opened a new log
I20260808 06:05:09.292194 14350 ts_tablet_manager.cc:1397] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d2678bd25fe44d98b1515f2188a9d777: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 06:05:09.292445 14384 ts_tablet_manager.cc:1918] T 8df38518f10b4caeaf1dde196e89305d P d784a288c65342a8a9b126e4ec8a618f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:05:09.292511 14384 log.cc:1196] T 8df38518f10b4caeaf1dde196e89305d P d784a288c65342a8a9b126e4ec8a618f: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-0/wal/wals/8df38518f10b4caeaf1dde196e89305d
I20260808 06:05:09.292629 14350 raft_consensus.cc:348] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d2678bd25fe44d98b1515f2188a9d777 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } }
I20260808 06:05:09.292739 14350 raft_consensus.cc:374] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d2678bd25fe44d98b1515f2188a9d777 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:05:09.292771 14350 raft_consensus.cc:729] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d2678bd25fe44d98b1515f2188a9d777 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2678bd25fe44d98b1515f2188a9d777, State: Initialized, Role: FOLLOWER
I20260808 06:05:09.292891 14350 consensus_queue.cc:260] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d2678bd25fe44d98b1515f2188a9d777 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } }
I20260808 06:05:09.293020 13973 tablet_service.cc:1467] Processing CreateTablet for tablet 73fa8a6c21d04ae98b2a7f0b856c57d8 (DEFAULT_TABLE table=test-table [id=b9cdcf0ded28487e99b4af14219983d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:05:09.293145 14350 ts_tablet_manager.cc:1428] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d2678bd25fe44d98b1515f2188a9d777: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 06:05:09.293267 13973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73fa8a6c21d04ae98b2a7f0b856c57d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:05:09.293612 14384 ts_tablet_manager.cc:1939] T 8df38518f10b4caeaf1dde196e89305d P d784a288c65342a8a9b126e4ec8a618f: Deleting consensus metadata
I20260808 06:05:09.295557 14347 tablet_bootstrap.cc:492] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P 0ecfb7f3474e44a6a9039eadb187ea9e: Bootstrap starting.
I20260808 06:05:09.295903 13518 catalog_manager.cc:4841] TS d784a288c65342a8a9b126e4ec8a618f (127.1.151.129:35179): tablet 8df38518f10b4caeaf1dde196e89305d (table test-table [id=255e1a8daf0e496eb5bec0069aa7c49d]) successfully deleted
I20260808 06:05:09.296468 14347 tablet_bootstrap.cc:654] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P 0ecfb7f3474e44a6a9039eadb187ea9e: Neither blocks nor log segments found. Creating new log.
I20260808 06:05:09.297583 14347 tablet_bootstrap.cc:492] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P 0ecfb7f3474e44a6a9039eadb187ea9e: No bootstrap required, opened a new log
I20260808 06:05:09.297652 14352 tablet_bootstrap.cc:492] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d784a288c65342a8a9b126e4ec8a618f: No bootstrap required, opened a new log
I20260808 06:05:09.297662 14347 ts_tablet_manager.cc:1397] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P 0ecfb7f3474e44a6a9039eadb187ea9e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 06:05:09.297726 14352 ts_tablet_manager.cc:1397] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d784a288c65342a8a9b126e4ec8a618f: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.001s
I20260808 06:05:09.298115 14347 raft_consensus.cc:348] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P 0ecfb7f3474e44a6a9039eadb187ea9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } }
I20260808 06:05:09.298136 14352 raft_consensus.cc:348] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d784a288c65342a8a9b126e4ec8a618f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } }
I20260808 06:05:09.298259 14352 raft_consensus.cc:374] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d784a288c65342a8a9b126e4ec8a618f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:05:09.298254 14347 raft_consensus.cc:374] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P 0ecfb7f3474e44a6a9039eadb187ea9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:05:09.298288 14352 raft_consensus.cc:729] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d784a288c65342a8a9b126e4ec8a618f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d784a288c65342a8a9b126e4ec8a618f, State: Initialized, Role: FOLLOWER
I20260808 06:05:09.298293 14347 raft_consensus.cc:729] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P 0ecfb7f3474e44a6a9039eadb187ea9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ecfb7f3474e44a6a9039eadb187ea9e, State: Initialized, Role: FOLLOWER
I20260808 06:05:09.298398 14352 consensus_queue.cc:260] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d784a288c65342a8a9b126e4ec8a618f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } }
I20260808 06:05:09.298408 14347 consensus_queue.cc:260] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P 0ecfb7f3474e44a6a9039eadb187ea9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } }
I20260808 06:05:09.298620 14352 ts_tablet_manager.cc:1428] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d784a288c65342a8a9b126e4ec8a618f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 06:05:09.298658 14347 ts_tablet_manager.cc:1428] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P 0ecfb7f3474e44a6a9039eadb187ea9e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 06:05:09.301003 14385 ts_tablet_manager.cc:1905] T 8df38518f10b4caeaf1dde196e89305d P 0ecfb7f3474e44a6a9039eadb187ea9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:05:09.303566 14385 ts_tablet_manager.cc:1918] T 8df38518f10b4caeaf1dde196e89305d P 0ecfb7f3474e44a6a9039eadb187ea9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:05:09.303676 14385 log.cc:1196] T 8df38518f10b4caeaf1dde196e89305d P 0ecfb7f3474e44a6a9039eadb187ea9e: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-1/wal/wals/8df38518f10b4caeaf1dde196e89305d
I20260808 06:05:09.303966 14385 ts_tablet_manager.cc:1939] T 8df38518f10b4caeaf1dde196e89305d P 0ecfb7f3474e44a6a9039eadb187ea9e: Deleting consensus metadata
I20260808 06:05:09.305032 13519 catalog_manager.cc:4841] TS 0ecfb7f3474e44a6a9039eadb187ea9e (127.1.151.130:44373): tablet 8df38518f10b4caeaf1dde196e89305d (table test-table [id=255e1a8daf0e496eb5bec0069aa7c49d]) successfully deleted
I20260808 06:05:09.438498 14358 raft_consensus.cc:482] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d784a288c65342a8a9b126e4ec8a618f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:05:09.438651 14358 raft_consensus.cc:504] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d784a288c65342a8a9b126e4ec8a618f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } }
I20260808 06:05:09.439628 14358 leader_election.cc:288] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d784a288c65342a8a9b126e4ec8a618f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ecfb7f3474e44a6a9039eadb187ea9e (127.1.151.130:44373), d2678bd25fe44d98b1515f2188a9d777 (127.1.151.131:35921)
I20260808 06:05:09.444846 14340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:09.439960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:48847 (local address 127.1.151.130:44373)
0808 06:05:09.440090 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:09.440095 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:09.440352 (+ 257us) server_negotiation.cc:407] Connection header received
0808 06:05:09.440489 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:09.440492 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:09.440546 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:09.440627 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:05:09.441023 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.441816 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.443535 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.444160 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.444286 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:09.444340 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:09.444623 (+ 283us) server_negotiation.cc:299] Negotiation successful
0808 06:05:09.444705 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 06:05:09.444846 14344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:09.439961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:39521 (local address 127.1.151.131:35921)
0808 06:05:09.440090 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:09.440094 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:09.440398 (+ 304us) server_negotiation.cc:407] Connection header received
0808 06:05:09.440489 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:09.440492 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:09.440546 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:09.440627 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:05:09.441018 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.441799 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.443535 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.444171 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.444286 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:09.444340 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:09.444675 (+ 335us) server_negotiation.cc:299] Negotiation successful
0808 06:05:09.444744 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 06:05:09.444886 14392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:09.439875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:44373 (local address 127.1.151.129:48847)
0808 06:05:09.440248 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:09.440264 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:09.440394 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:09.440660 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:09.440663 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:09.440702 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:05:09.440905 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:09.440914 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.441924 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.441928 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:09.443395 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:09.443405 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.444320 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.444324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:09.444486 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:09.444523 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 06:05:09.444605 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 06:05:09.444703 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":210,"spinlock_wait_cycles":16512,"thread_start_us":108,"threads_started":1}
I20260808 06:05:09.444886 14391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:09.439875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:35921 (local address 127.1.151.129:39521)
0808 06:05:09.440300 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:09.440311 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:09.440400 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:09.440660 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:09.440663 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:09.440703 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:05:09.440913 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:09.440919 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.441924 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.441928 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:09.443391 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:09.443405 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.444320 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.444324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:09.444482 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:09.444582 (+ 100us) client_negotiation.cc:769] Sending connection context
0808 06:05:09.444647 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 06:05:09.444722 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":154,"spinlock_wait_cycles":10368,"thread_start_us":83,"threads_started":1}
I20260808 06:05:09.445432 14004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73fa8a6c21d04ae98b2a7f0b856c57d8" candidate_uuid: "d784a288c65342a8a9b126e4ec8a618f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" is_pre_election: true
I20260808 06:05:09.445694 14004 raft_consensus.cc:2454] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P 0ecfb7f3474e44a6a9039eadb187ea9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d784a288c65342a8a9b126e4ec8a618f in term 0.
I20260808 06:05:09.445806 14236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73fa8a6c21d04ae98b2a7f0b856c57d8" candidate_uuid: "d784a288c65342a8a9b126e4ec8a618f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2678bd25fe44d98b1515f2188a9d777" is_pre_election: true
I20260808 06:05:09.445955 14236 raft_consensus.cc:2454] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d2678bd25fe44d98b1515f2188a9d777 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d784a288c65342a8a9b126e4ec8a618f in term 0.
I20260808 06:05:09.446126 13665 leader_election.cc:302] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d784a288c65342a8a9b126e4ec8a618f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ecfb7f3474e44a6a9039eadb187ea9e, d784a288c65342a8a9b126e4ec8a618f; no voters:
I20260808 06:05:09.446439 14358 raft_consensus.cc:2781] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d784a288c65342a8a9b126e4ec8a618f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 06:05:09.446498 14358 raft_consensus.cc:482] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d784a288c65342a8a9b126e4ec8a618f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 06:05:09.446524 14358 raft_consensus.cc:3037] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d784a288c65342a8a9b126e4ec8a618f [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:05:09.447506 14358 raft_consensus.cc:504] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d784a288c65342a8a9b126e4ec8a618f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } }
I20260808 06:05:09.447850 14358 leader_election.cc:288] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d784a288c65342a8a9b126e4ec8a618f [CANDIDATE]: Term 1 election: Requested vote from peers 0ecfb7f3474e44a6a9039eadb187ea9e (127.1.151.130:44373), d2678bd25fe44d98b1515f2188a9d777 (127.1.151.131:35921)
I20260808 06:05:09.448196 14236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73fa8a6c21d04ae98b2a7f0b856c57d8" candidate_uuid: "d784a288c65342a8a9b126e4ec8a618f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2678bd25fe44d98b1515f2188a9d777"
I20260808 06:05:09.448237 14004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73fa8a6c21d04ae98b2a7f0b856c57d8" candidate_uuid: "d784a288c65342a8a9b126e4ec8a618f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e"
I20260808 06:05:09.448297 14236 raft_consensus.cc:3037] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d2678bd25fe44d98b1515f2188a9d777 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:05:09.448357 14004 raft_consensus.cc:3037] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P 0ecfb7f3474e44a6a9039eadb187ea9e [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:05:09.449169 14236 raft_consensus.cc:2454] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d2678bd25fe44d98b1515f2188a9d777 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d784a288c65342a8a9b126e4ec8a618f in term 1.
I20260808 06:05:09.449353 14004 raft_consensus.cc:2454] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P 0ecfb7f3474e44a6a9039eadb187ea9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d784a288c65342a8a9b126e4ec8a618f in term 1.
I20260808 06:05:09.449462 13663 leader_election.cc:302] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d784a288c65342a8a9b126e4ec8a618f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2678bd25fe44d98b1515f2188a9d777, d784a288c65342a8a9b126e4ec8a618f; no voters:
I20260808 06:05:09.449645 14358 raft_consensus.cc:2781] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d784a288c65342a8a9b126e4ec8a618f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:05:09.449857 14358 raft_consensus.cc:686] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d784a288c65342a8a9b126e4ec8a618f [term 1 LEADER]: Becoming Leader. State: Replica: d784a288c65342a8a9b126e4ec8a618f, State: Running, Role: LEADER
I20260808 06:05:09.450088 14358 consensus_queue.cc:237] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d784a288c65342a8a9b126e4ec8a618f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } }
I20260808 06:05:09.452287 13536 catalog_manager.cc:5495] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d784a288c65342a8a9b126e4ec8a618f reported cstate change: term changed from 0 to 1, leader changed from <none> to d784a288c65342a8a9b126e4ec8a618f (127.1.151.129). New cstate: current_term: 1 leader_uuid: "d784a288c65342a8a9b126e4ec8a618f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } health_report { overall_health: UNKNOWN } } }
Aug 08 06:05:09 dist-test-slave-zj8q krb5kdc[13476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169108, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.190@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM
I20260808 06:05:09.499575 14403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:09.491786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:41443 (local address 127.0.0.1:47742)
0808 06:05:09.492021 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:09.492034 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:09.492163 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:09.492378 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:09.492380 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:09.492783 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:09.492874 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:09.492881 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.493588 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.493591 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:09.494145 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:09.494193 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.494609 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.494612 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:09.494705 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:09.495238 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:09.495250 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:09.496630 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:09.498064 (+ 1434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:09.498072 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:09.498077 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:09.498341 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:09.498586 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:09.498588 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:09.498589 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:09.498682 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:09.499032 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:09.499039 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:09.499140 (+ 101us) client_negotiation.cc:769] Sending connection context
0808 06:05:09.499235 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 06:05:09.499383 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":2,"thread_start_us":86,"threads_started":1}
I20260808 06:05:09.499574 13610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:09.491842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47742 (local address 127.1.151.190:41443)
0808 06:05:09.491958 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:09.491962 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:09.492091 (+ 129us) server_negotiation.cc:407] Connection header received
0808 06:05:09.492248 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:09.492251 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:09.492289 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:09.492350 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:09.492983 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.493493 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.494284 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.494512 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.494613 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:09.496767 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:09.496770 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:09.496772 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:09.496794 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:09.497968 (+ 1174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:09.498414 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:09.498417 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:09.498419 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:09.498508 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:09.498764 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:09.498768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:09.498770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:09.498936 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:09.499020 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:09.499292 (+ 272us) server_negotiation.cc:299] Negotiation successful
0808 06:05:09.499446 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":75,"server-negotiator.queue_time_us":36}
I20260808 06:05:09.503260 13536 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:47742:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 06:05:09.503984 13536 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 06:05:09.508548 13750 tablet_service.cc:1467] Processing CreateTablet for tablet fdaf02e2240c4f08a0c7fcb27d87f7c5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dace31ae426d4ee181de1197ac9037ee]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 06:05:09.508610 13973 tablet_service.cc:1467] Processing CreateTablet for tablet fdaf02e2240c4f08a0c7fcb27d87f7c5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dace31ae426d4ee181de1197ac9037ee]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 06:05:09.508708 14206 tablet_service.cc:1467] Processing CreateTablet for tablet fdaf02e2240c4f08a0c7fcb27d87f7c5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dace31ae426d4ee181de1197ac9037ee]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 06:05:09.509003 13973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdaf02e2240c4f08a0c7fcb27d87f7c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:05:09.509006 13750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdaf02e2240c4f08a0c7fcb27d87f7c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:05:09.509027 14206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdaf02e2240c4f08a0c7fcb27d87f7c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:05:09.511905 14352 tablet_bootstrap.cc:492] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d784a288c65342a8a9b126e4ec8a618f: Bootstrap starting.
I20260808 06:05:09.511905 14350 tablet_bootstrap.cc:492] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d2678bd25fe44d98b1515f2188a9d777: Bootstrap starting.
I20260808 06:05:09.511981 14347 tablet_bootstrap.cc:492] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P 0ecfb7f3474e44a6a9039eadb187ea9e: Bootstrap starting.
I20260808 06:05:09.512703 14347 tablet_bootstrap.cc:654] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P 0ecfb7f3474e44a6a9039eadb187ea9e: Neither blocks nor log segments found. Creating new log.
I20260808 06:05:09.512831 14352 tablet_bootstrap.cc:654] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d784a288c65342a8a9b126e4ec8a618f: Neither blocks nor log segments found. Creating new log.
I20260808 06:05:09.512831 14350 tablet_bootstrap.cc:654] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d2678bd25fe44d98b1515f2188a9d777: Neither blocks nor log segments found. Creating new log.
I20260808 06:05:09.513748 14347 tablet_bootstrap.cc:492] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P 0ecfb7f3474e44a6a9039eadb187ea9e: No bootstrap required, opened a new log
I20260808 06:05:09.513842 14347 ts_tablet_manager.cc:1397] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P 0ecfb7f3474e44a6a9039eadb187ea9e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 06:05:09.514278 14352 tablet_bootstrap.cc:492] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d784a288c65342a8a9b126e4ec8a618f: No bootstrap required, opened a new log
I20260808 06:05:09.514276 14347 raft_consensus.cc:348] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P 0ecfb7f3474e44a6a9039eadb187ea9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } }
I20260808 06:05:09.514353 14352 ts_tablet_manager.cc:1397] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d784a288c65342a8a9b126e4ec8a618f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 06:05:09.514395 14347 raft_consensus.cc:374] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P 0ecfb7f3474e44a6a9039eadb187ea9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:05:09.514433 14347 raft_consensus.cc:729] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P 0ecfb7f3474e44a6a9039eadb187ea9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ecfb7f3474e44a6a9039eadb187ea9e, State: Initialized, Role: FOLLOWER
I20260808 06:05:09.514561 14347 consensus_queue.cc:260] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P 0ecfb7f3474e44a6a9039eadb187ea9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } }
I20260808 06:05:09.514782 14347 ts_tablet_manager.cc:1428] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P 0ecfb7f3474e44a6a9039eadb187ea9e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 06:05:09.514760 14352 raft_consensus.cc:348] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d784a288c65342a8a9b126e4ec8a618f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } }
I20260808 06:05:09.514842 14352 raft_consensus.cc:374] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d784a288c65342a8a9b126e4ec8a618f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:05:09.514880 14352 raft_consensus.cc:729] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d784a288c65342a8a9b126e4ec8a618f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d784a288c65342a8a9b126e4ec8a618f, State: Initialized, Role: FOLLOWER
I20260808 06:05:09.514966 14379 tablet_replica.cc:439] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P 0ecfb7f3474e44a6a9039eadb187ea9e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } } }
I20260808 06:05:09.515012 14352 consensus_queue.cc:260] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d784a288c65342a8a9b126e4ec8a618f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } }
I20260808 06:05:09.515090 14379 tablet_replica.cc:442] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P 0ecfb7f3474e44a6a9039eadb187ea9e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 06:05:09.515260 14352 ts_tablet_manager.cc:1428] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d784a288c65342a8a9b126e4ec8a618f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 06:05:09.515228 14358 tablet_replica.cc:439] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d784a288c65342a8a9b126e4ec8a618f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } } }
I20260808 06:05:09.515344 14358 tablet_replica.cc:442] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d784a288c65342a8a9b126e4ec8a618f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 06:05:09.516172 14350 tablet_bootstrap.cc:492] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d2678bd25fe44d98b1515f2188a9d777: No bootstrap required, opened a new log
I20260808 06:05:09.516254 14350 ts_tablet_manager.cc:1397] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d2678bd25fe44d98b1515f2188a9d777: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 06:05:09.516607 14350 raft_consensus.cc:348] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d2678bd25fe44d98b1515f2188a9d777 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } }
I20260808 06:05:09.516709 14350 raft_consensus.cc:374] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d2678bd25fe44d98b1515f2188a9d777 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:05:09.516741 14350 raft_consensus.cc:729] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d2678bd25fe44d98b1515f2188a9d777 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2678bd25fe44d98b1515f2188a9d777, State: Initialized, Role: FOLLOWER
I20260808 06:05:09.516851 14350 consensus_queue.cc:260] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d2678bd25fe44d98b1515f2188a9d777 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } }
I20260808 06:05:09.517076 14350 ts_tablet_manager.cc:1428] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d2678bd25fe44d98b1515f2188a9d777: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 06:05:09.517266 14357 tablet_replica.cc:439] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d2678bd25fe44d98b1515f2188a9d777: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } } }
I20260808 06:05:09.517379 14357 tablet_replica.cc:442] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d2678bd25fe44d98b1515f2188a9d777: This TxnStatusTablet replica's current role is: FOLLOWER
W20260808 06:05:09.525180 13854 tablet.cc:2369] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d784a288c65342a8a9b126e4ec8a618f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 06:05:09.726943 14357 raft_consensus.cc:482] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d2678bd25fe44d98b1515f2188a9d777 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:05:09.727098 14357 raft_consensus.cc:504] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d2678bd25fe44d98b1515f2188a9d777 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } }
I20260808 06:05:09.728338 14357 leader_election.cc:288] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d2678bd25fe44d98b1515f2188a9d777 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d784a288c65342a8a9b126e4ec8a618f (127.1.151.129:35179), 0ecfb7f3474e44a6a9039eadb187ea9e (127.1.151.130:44373)
I20260808 06:05:09.733178 14340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:09.728682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:43671 (local address 127.1.151.130:44373)
0808 06:05:09.728818 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:09.728823 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:09.729186 (+ 363us) server_negotiation.cc:407] Connection header received
0808 06:05:09.729387 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:09.729390 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:09.729443 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:09.729505 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:05:09.729763 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.730363 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.732067 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.732661 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.732781 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:09.732837 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:09.732989 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 06:05:09.733071 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 06:05:09.733209 14409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:09.728544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:44373 (local address 127.1.151.131:43671)
0808 06:05:09.729069 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:09.729103 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:09.729257 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:09.729538 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:09.729541 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:09.729557 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:05:09.729662 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:09.729667 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.730489 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.730492 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:09.731934 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:09.731947 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.732770 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.732773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:09.732862 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:09.732890 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 06:05:09.732959 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 06:05:09.733032 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"mutex_wait_us":259,"thread_start_us":83,"threads_started":1}
I20260808 06:05:09.733856 14004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdaf02e2240c4f08a0c7fcb27d87f7c5" candidate_uuid: "d2678bd25fe44d98b1515f2188a9d777" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" is_pre_election: true
I20260808 06:05:09.734040 14004 raft_consensus.cc:2454] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P 0ecfb7f3474e44a6a9039eadb187ea9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2678bd25fe44d98b1515f2188a9d777 in term 0.
I20260808 06:05:09.734407 14407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:09.728544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:35179 (local address 127.1.151.131:41909)
0808 06:05:09.728831 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:09.728845 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:09.728959 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:09.729293 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:09.729296 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:09.729315 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:05:09.729447 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:09.729455 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.730489 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.730492 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:09.731934 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:09.731947 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.733901 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.733905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:09.734001 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:09.734025 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 06:05:09.734097 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 06:05:09.734208 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":145,"thread_start_us":99,"threads_started":1}
I20260808 06:05:09.734486 14128 leader_election.cc:302] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d2678bd25fe44d98b1515f2188a9d777 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ecfb7f3474e44a6a9039eadb187ea9e, d2678bd25fe44d98b1515f2188a9d777; no voters:
I20260808 06:05:09.734427 14408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:09.728665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:41909 (local address 127.1.151.129:35179)
0808 06:05:09.728929 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:09.728933 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:09.728951 (+ 18us) server_negotiation.cc:407] Connection header received
0808 06:05:09.729070 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:09.729074 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:09.729155 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:09.729240 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:05:09.729583 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.730394 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.732897 (+ 2503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.733781 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.733907 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:09.733956 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:09.734214 (+ 258us) server_negotiation.cc:299] Negotiation successful
0808 06:05:09.734295 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":58,"threads_started":1}
I20260808 06:05:09.734812 14357 raft_consensus.cc:2781] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d2678bd25fe44d98b1515f2188a9d777 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 06:05:09.734876 14357 raft_consensus.cc:482] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d2678bd25fe44d98b1515f2188a9d777 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 06:05:09.734915 14357 raft_consensus.cc:3037] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d2678bd25fe44d98b1515f2188a9d777 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:05:09.735073 13779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdaf02e2240c4f08a0c7fcb27d87f7c5" candidate_uuid: "d2678bd25fe44d98b1515f2188a9d777" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d784a288c65342a8a9b126e4ec8a618f" is_pre_election: true
I20260808 06:05:09.735208 13779 raft_consensus.cc:2454] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d784a288c65342a8a9b126e4ec8a618f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2678bd25fe44d98b1515f2188a9d777 in term 0.
I20260808 06:05:09.736095 14357 raft_consensus.cc:504] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d2678bd25fe44d98b1515f2188a9d777 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } }
I20260808 06:05:09.736446 14357 leader_election.cc:288] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d2678bd25fe44d98b1515f2188a9d777 [CANDIDATE]: Term 1 election: Requested vote from peers d784a288c65342a8a9b126e4ec8a618f (127.1.151.129:35179), 0ecfb7f3474e44a6a9039eadb187ea9e (127.1.151.130:44373)
I20260808 06:05:09.736759 13779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdaf02e2240c4f08a0c7fcb27d87f7c5" candidate_uuid: "d2678bd25fe44d98b1515f2188a9d777" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d784a288c65342a8a9b126e4ec8a618f"
I20260808 06:05:09.736758 14004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdaf02e2240c4f08a0c7fcb27d87f7c5" candidate_uuid: "d2678bd25fe44d98b1515f2188a9d777" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e"
I20260808 06:05:09.736882 13779 raft_consensus.cc:3037] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d784a288c65342a8a9b126e4ec8a618f [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:05:09.736881 14004 raft_consensus.cc:3037] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P 0ecfb7f3474e44a6a9039eadb187ea9e [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:05:09.737967 14004 raft_consensus.cc:2454] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P 0ecfb7f3474e44a6a9039eadb187ea9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2678bd25fe44d98b1515f2188a9d777 in term 1.
I20260808 06:05:09.737967 13779 raft_consensus.cc:2454] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d784a288c65342a8a9b126e4ec8a618f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2678bd25fe44d98b1515f2188a9d777 in term 1.
I20260808 06:05:09.738265 14128 leader_election.cc:302] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d2678bd25fe44d98b1515f2188a9d777 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ecfb7f3474e44a6a9039eadb187ea9e, d2678bd25fe44d98b1515f2188a9d777; no voters:
I20260808 06:05:09.738425 14357 raft_consensus.cc:2781] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d2678bd25fe44d98b1515f2188a9d777 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:05:09.738590 14357 raft_consensus.cc:686] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d2678bd25fe44d98b1515f2188a9d777 [term 1 LEADER]: Becoming Leader. State: Replica: d2678bd25fe44d98b1515f2188a9d777, State: Running, Role: LEADER
I20260808 06:05:09.738822 14357 consensus_queue.cc:237] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d2678bd25fe44d98b1515f2188a9d777 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } }
I20260808 06:05:09.740021 14411 tablet_replica.cc:439] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d2678bd25fe44d98b1515f2188a9d777: TxnStatusTablet state changed. Reason: New leader d2678bd25fe44d98b1515f2188a9d777. Latest consensus state: current_term: 1 leader_uuid: "d2678bd25fe44d98b1515f2188a9d777" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } } }
I20260808 06:05:09.740172 14411 tablet_replica.cc:442] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d2678bd25fe44d98b1515f2188a9d777: This TxnStatusTablet replica's current role is: LEADER
I20260808 06:05:09.741209 13536 catalog_manager.cc:5495] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d2678bd25fe44d98b1515f2188a9d777 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2678bd25fe44d98b1515f2188a9d777 (127.1.151.131). New cstate: current_term: 1 leader_uuid: "d2678bd25fe44d98b1515f2188a9d777" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } health_report { overall_health: HEALTHY } } }
I20260808 06:05:09.746331 14415 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 06:05:09.746505 14415 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 06:05:09.756389 14403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:09.752440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:35921 (local address 127.0.0.1:47230)
0808 06:05:09.752593 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:09.752610 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:09.752714 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:09.753010 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:09.753012 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:09.753018 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 06:05:09.753127 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:09.753132 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.754184 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.754188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:09.754801 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:09.754811 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.755361 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.755364 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:09.755423 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:09.755464 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 06:05:09.756160 (+ 696us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 06:05:09.756168 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 06:05:09.756213 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 06:05:09.756285 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 06:05:09.756420 14344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:09.752531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47230 (local address 127.1.151.131:35921)
0808 06:05:09.752647 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:09.752651 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:09.752667 (+ 16us) server_negotiation.cc:407] Connection header received
0808 06:05:09.752841 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:09.752845 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:09.752905 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:09.753009 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:05:09.753243 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.754013 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.754918 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.755265 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.755375 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:09.755585 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:05:09.756044 (+ 459us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:05:09.756173 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:09.756229 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 06:05:09.756286 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 06:05:09.758935 13779 raft_consensus.cc:1261] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d784a288c65342a8a9b126e4ec8a618f [term 1 FOLLOWER]: Refusing update from remote peer d2678bd25fe44d98b1515f2188a9d777: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:05:09.759294 14004 raft_consensus.cc:1261] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P 0ecfb7f3474e44a6a9039eadb187ea9e [term 1 FOLLOWER]: Refusing update from remote peer d2678bd25fe44d98b1515f2188a9d777: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:05:09.759403 14357 consensus_queue.cc:1035] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d2678bd25fe44d98b1515f2188a9d777 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:05:09.759765 14357 consensus_queue.cc:1035] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d2678bd25fe44d98b1515f2188a9d777 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:05:09.760980 14358 tablet_replica.cc:439] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d784a288c65342a8a9b126e4ec8a618f: TxnStatusTablet state changed. Reason: New leader d2678bd25fe44d98b1515f2188a9d777. Latest consensus state: current_term: 1 leader_uuid: "d2678bd25fe44d98b1515f2188a9d777" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } } }
I20260808 06:05:09.761089 14358 tablet_replica.cc:442] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d784a288c65342a8a9b126e4ec8a618f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 06:05:09.761233 14379 tablet_replica.cc:439] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P 0ecfb7f3474e44a6a9039eadb187ea9e: TxnStatusTablet state changed. Reason: New leader d2678bd25fe44d98b1515f2188a9d777. Latest consensus state: current_term: 1 leader_uuid: "d2678bd25fe44d98b1515f2188a9d777" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } } }
I20260808 06:05:09.761332 14379 tablet_replica.cc:442] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P 0ecfb7f3474e44a6a9039eadb187ea9e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 06:05:09.763453 14411 tablet_replica.cc:439] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d2678bd25fe44d98b1515f2188a9d777: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d2678bd25fe44d98b1515f2188a9d777" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } } }
I20260808 06:05:09.763612 14411 tablet_replica.cc:442] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d2678bd25fe44d98b1515f2188a9d777: This TxnStatusTablet replica's current role is: LEADER
I20260808 06:05:09.764451 14379 tablet_replica.cc:439] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P 0ecfb7f3474e44a6a9039eadb187ea9e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d2678bd25fe44d98b1515f2188a9d777" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } } }
I20260808 06:05:09.764549 14379 tablet_replica.cc:442] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P 0ecfb7f3474e44a6a9039eadb187ea9e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 06:05:09.771574 14357 tablet_replica.cc:439] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d2678bd25fe44d98b1515f2188a9d777: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d2678bd25fe44d98b1515f2188a9d777" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } } }
I20260808 06:05:09.771679 14357 tablet_replica.cc:442] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d2678bd25fe44d98b1515f2188a9d777: This TxnStatusTablet replica's current role is: LEADER
I20260808 06:05:09.771727 14408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:09.767673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41278 (local address 127.1.151.129:35179)
0808 06:05:09.767789 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:09.767793 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:09.767942 (+ 149us) server_negotiation.cc:407] Connection header received
0808 06:05:09.768069 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:09.768072 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:09.768141 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:09.768243 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:05:09.768421 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.769196 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.769977 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.770346 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.770443 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:09.770743 (+ 300us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:05:09.771259 (+ 516us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:05:09.771360 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:09.771527 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 06:05:09.771592 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 06:05:09.772526 14358 tablet_replica.cc:439] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d784a288c65342a8a9b126e4ec8a618f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d2678bd25fe44d98b1515f2188a9d777" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } } }
I20260808 06:05:09.772639 14358 tablet_replica.cc:442] T fdaf02e2240c4f08a0c7fcb27d87f7c5 P d784a288c65342a8a9b126e4ec8a618f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 06:05:09.783259 14344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:09.780135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:45581 (local address 127.1.151.131:35921)
0808 06:05:09.780249 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:09.780252 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:09.780265 (+ 13us) server_negotiation.cc:407] Connection header received
0808 06:05:09.780363 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:09.780365 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:09.780402 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:09.780459 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:05:09.780700 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.781174 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.782378 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.782580 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.782648 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:09.782922 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:05:09.782984 (+ 62us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:05:09.783026 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:09.783117 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 06:05:09.783164 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 06:05:09.783361 14429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:09.779873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:35921 (local address 127.1.151.129:45581)
0808 06:05:09.780174 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:09.780189 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:09.780280 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:09.780481 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:09.780483 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:09.780490 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 06:05:09.780610 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:09.780617 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.781258 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.781261 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:09.782275 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:09.782286 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.782664 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.782668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:09.782763 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:09.782813 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 06:05:09.783041 (+ 228us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 06:05:09.783046 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 06:05:09.783114 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 06:05:09.783191 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":96,"threads_started":1}
I20260808 06:05:09.789633 14236 raft_consensus.cc:1261] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d2678bd25fe44d98b1515f2188a9d777 [term 1 FOLLOWER]: Refusing update from remote peer d784a288c65342a8a9b126e4ec8a618f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:05:09.789734 14004 raft_consensus.cc:1261] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P 0ecfb7f3474e44a6a9039eadb187ea9e [term 1 FOLLOWER]: Refusing update from remote peer d784a288c65342a8a9b126e4ec8a618f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:05:09.790113 14393 consensus_queue.cc:1035] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d784a288c65342a8a9b126e4ec8a618f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:05:09.790328 14358 consensus_queue.cc:1035] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d784a288c65342a8a9b126e4ec8a618f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:05:09.796399 14367 mvcc.cc:205] Tried to move back new op lower bound from 7316148673694253056 to 7316148672309874688. Current Snapshot: MvccSnapshot[applied={T|T < 7316148673694253056}]
I20260808 06:05:09.811878 14408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:09.808386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:41975 (local address 127.1.151.129:35179)
0808 06:05:09.808508 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:09.808513 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:09.808774 (+ 261us) server_negotiation.cc:407] Connection header received
0808 06:05:09.808866 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:09.808869 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:09.808920 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:09.808990 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:05:09.809309 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.809922 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.810890 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.811115 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.811202 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:09.811476 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:05:09.811548 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:05:09.811606 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:09.811693 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 06:05:09.811741 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 06:05:09.811906 14436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:09.808308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:35179 (local address 127.1.151.131:41975)
0808 06:05:09.808680 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:09.808691 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:09.808787 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:09.809041 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:09.809045 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:09.809054 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 06:05:09.809190 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:09.809197 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.810025 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.810029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:09.810775 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:09.810787 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.811212 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.811215 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:09.811314 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:09.811357 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 06:05:09.811619 (+ 262us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 06:05:09.811625 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 06:05:09.811683 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 06:05:09.811761 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":97,"threads_started":1}
I20260808 06:05:09.833025 13535 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:05:09.833173 13535 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:05:09.835132 13535 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f5f0153a8bbb4a819b1bb2eebff8ca4f: Sending DeleteTablet for 3 replicas of tablet 73fa8a6c21d04ae98b2a7f0b856c57d8
I20260808 06:05:09.835713 13750 tablet_service.cc:1514] Processing DeleteTablet for tablet 73fa8a6c21d04ae98b2a7f0b856c57d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:05:09 UTC) from {username='oryx', principal='oryx/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:41262
I20260808 06:05:09.835794 13973 tablet_service.cc:1514] Processing DeleteTablet for tablet 73fa8a6c21d04ae98b2a7f0b856c57d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:05:09 UTC) from {username='oryx', principal='oryx/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:38722
I20260808 06:05:09.836395 14206 tablet_service.cc:1514] Processing DeleteTablet for tablet 73fa8a6c21d04ae98b2a7f0b856c57d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:05:09 UTC) from {username='oryx', principal='oryx/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:47214
I20260808 06:05:09.836745 14445 tablet_replica.cc:330] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d784a288c65342a8a9b126e4ec8a618f: stopping tablet replica
I20260808 06:05:09.836874 14445 raft_consensus.cc:2229] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d784a288c65342a8a9b126e4ec8a618f [term 1 LEADER]: Raft consensus shutting down.
I20260808 06:05:09.837158 14445 raft_consensus.cc:2258] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d784a288c65342a8a9b126e4ec8a618f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:05:09.837394 14447 tablet_replica.cc:330] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P 0ecfb7f3474e44a6a9039eadb187ea9e: stopping tablet replica
I20260808 06:05:09.837514 14447 raft_consensus.cc:2229] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P 0ecfb7f3474e44a6a9039eadb187ea9e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:05:09.837617 14447 raft_consensus.cc:2258] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P 0ecfb7f3474e44a6a9039eadb187ea9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:05:09.838315 14445 ts_tablet_manager.cc:1905] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d784a288c65342a8a9b126e4ec8a618f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:05:09.841336 14445 ts_tablet_manager.cc:1918] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d784a288c65342a8a9b126e4ec8a618f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 06:05:09.841420 14445 log.cc:1196] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d784a288c65342a8a9b126e4ec8a618f: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-0/wal/wals/73fa8a6c21d04ae98b2a7f0b856c57d8
I20260808 06:05:09.838357 14447 ts_tablet_manager.cc:1905] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P 0ecfb7f3474e44a6a9039eadb187ea9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:05:09.841675 14445 ts_tablet_manager.cc:1939] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d784a288c65342a8a9b126e4ec8a618f: Deleting consensus metadata
I20260808 06:05:09.842427 14448 tablet_replica.cc:330] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d2678bd25fe44d98b1515f2188a9d777: stopping tablet replica
I20260808 06:05:09.842406 13518 catalog_manager.cc:4841] TS d784a288c65342a8a9b126e4ec8a618f (127.1.151.129:35179): tablet 73fa8a6c21d04ae98b2a7f0b856c57d8 (table test-table [id=b9cdcf0ded28487e99b4af14219983d8]) successfully deleted
I20260808 06:05:09.842551 14448 raft_consensus.cc:2229] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d2678bd25fe44d98b1515f2188a9d777 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:05:09.842670 14448 raft_consensus.cc:2258] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d2678bd25fe44d98b1515f2188a9d777 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:05:09.843248 14448 ts_tablet_manager.cc:1905] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d2678bd25fe44d98b1515f2188a9d777: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:05:09.845847 14448 ts_tablet_manager.cc:1918] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d2678bd25fe44d98b1515f2188a9d777: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 06:05:09.845927 14448 log.cc:1196] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d2678bd25fe44d98b1515f2188a9d777: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-2/wal/wals/73fa8a6c21d04ae98b2a7f0b856c57d8
I20260808 06:05:09.846199 14448 ts_tablet_manager.cc:1939] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P d2678bd25fe44d98b1515f2188a9d777: Deleting consensus metadata
I20260808 06:05:09.846879 13518 catalog_manager.cc:4841] TS d2678bd25fe44d98b1515f2188a9d777 (127.1.151.131:35921): tablet 73fa8a6c21d04ae98b2a7f0b856c57d8 (table test-table [id=b9cdcf0ded28487e99b4af14219983d8]) successfully deleted
I20260808 06:05:09.847776 14447 ts_tablet_manager.cc:1918] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P 0ecfb7f3474e44a6a9039eadb187ea9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 06:05:09.847857 14447 log.cc:1196] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P 0ecfb7f3474e44a6a9039eadb187ea9e: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-1/wal/wals/73fa8a6c21d04ae98b2a7f0b856c57d8
I20260808 06:05:09.848098 14447 ts_tablet_manager.cc:1939] T 73fa8a6c21d04ae98b2a7f0b856c57d8 P 0ecfb7f3474e44a6a9039eadb187ea9e: Deleting consensus metadata
I20260808 06:05:09.848748 13519 catalog_manager.cc:4841] TS 0ecfb7f3474e44a6a9039eadb187ea9e (127.1.151.130:44373): tablet 73fa8a6c21d04ae98b2a7f0b856c57d8 (table test-table [id=b9cdcf0ded28487e99b4af14219983d8]) successfully deleted
Aug 08 06:05:09 dist-test-slave-zj8q krb5kdc[13476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 06:05:09 dist-test-slave-zj8q krb5kdc[13476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:09 dist-test-slave-zj8q krb5kdc[13476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 06:05:09.868769 13610 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:09.862868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47754 (local address 127.1.151.190:41443)
0808 06:05:09.863009 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:09.863014 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:09.863187 (+ 173us) server_negotiation.cc:407] Connection header received
0808 06:05:09.863314 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:09.863317 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:09.863366 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:09.863430 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:09.864111 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.864631 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.865477 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.865717 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.865806 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:09.868670 (+ 2864us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47754: BlockingRecv error: recv got EOF from 127.0.0.1:47754 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 08 06:05:09 dist-test-slave-zj8q krb5kdc[13476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM
I20260808 06:05:09.878643 13610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:09.870466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47770 (local address 127.1.151.190:41443)
0808 06:05:09.870567 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:09.870571 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:09.870931 (+ 360us) server_negotiation.cc:407] Connection header received
0808 06:05:09.871029 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:09.871032 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:09.871073 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:09.871134 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:09.871685 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.872161 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.872890 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:09.873101 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:09.873186 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:09.875299 (+ 2113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:09.875302 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:09.875304 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:09.875327 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:09.876690 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:09.877159 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:09.877163 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:09.877165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:09.877243 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:09.877572 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:09.877575 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:09.877577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:09.877750 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:09.878180 (+ 430us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:09.878382 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 06:05:09.878489 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 06:05:09.881064 13535 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47770:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 06:05:09.881501 13535 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 06:05:09.886528 13973 tablet_service.cc:1467] Processing CreateTablet for tablet 376c349bf9a74fb28ea7a0e46e2c7066 (DEFAULT_TABLE table=test-table [id=4ec32127323741ee866356a7e943cd4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:05:09.886396 13750 tablet_service.cc:1467] Processing CreateTablet for tablet 376c349bf9a74fb28ea7a0e46e2c7066 (DEFAULT_TABLE table=test-table [id=4ec32127323741ee866356a7e943cd4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:05:09.886865 13973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 376c349bf9a74fb28ea7a0e46e2c7066. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:05:09.886938 13750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 376c349bf9a74fb28ea7a0e46e2c7066. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:05:09.889559 14352 tablet_bootstrap.cc:492] T 376c349bf9a74fb28ea7a0e46e2c7066 P d784a288c65342a8a9b126e4ec8a618f: Bootstrap starting.
I20260808 06:05:09.890345 14352 tablet_bootstrap.cc:654] T 376c349bf9a74fb28ea7a0e46e2c7066 P d784a288c65342a8a9b126e4ec8a618f: Neither blocks nor log segments found. Creating new log.
I20260808 06:05:09.890892 14347 tablet_bootstrap.cc:492] T 376c349bf9a74fb28ea7a0e46e2c7066 P 0ecfb7f3474e44a6a9039eadb187ea9e: Bootstrap starting.
I20260808 06:05:09.891592 14347 tablet_bootstrap.cc:654] T 376c349bf9a74fb28ea7a0e46e2c7066 P 0ecfb7f3474e44a6a9039eadb187ea9e: Neither blocks nor log segments found. Creating new log.
I20260808 06:05:09.892175 14206 tablet_service.cc:1467] Processing CreateTablet for tablet 376c349bf9a74fb28ea7a0e46e2c7066 (DEFAULT_TABLE table=test-table [id=4ec32127323741ee866356a7e943cd4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:05:09.892405 14206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 376c349bf9a74fb28ea7a0e46e2c7066. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:05:09.894101 14350 tablet_bootstrap.cc:492] T 376c349bf9a74fb28ea7a0e46e2c7066 P d2678bd25fe44d98b1515f2188a9d777: Bootstrap starting.
I20260808 06:05:09.895007 14350 tablet_bootstrap.cc:654] T 376c349bf9a74fb28ea7a0e46e2c7066 P d2678bd25fe44d98b1515f2188a9d777: Neither blocks nor log segments found. Creating new log.
I20260808 06:05:09.895299 14347 tablet_bootstrap.cc:492] T 376c349bf9a74fb28ea7a0e46e2c7066 P 0ecfb7f3474e44a6a9039eadb187ea9e: No bootstrap required, opened a new log
I20260808 06:05:09.895377 14347 ts_tablet_manager.cc:1397] T 376c349bf9a74fb28ea7a0e46e2c7066 P 0ecfb7f3474e44a6a9039eadb187ea9e: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260808 06:05:09.895717 14347 raft_consensus.cc:348] T 376c349bf9a74fb28ea7a0e46e2c7066 P 0ecfb7f3474e44a6a9039eadb187ea9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } }
I20260808 06:05:09.895807 14347 raft_consensus.cc:374] T 376c349bf9a74fb28ea7a0e46e2c7066 P 0ecfb7f3474e44a6a9039eadb187ea9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:05:09.895828 14347 raft_consensus.cc:729] T 376c349bf9a74fb28ea7a0e46e2c7066 P 0ecfb7f3474e44a6a9039eadb187ea9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ecfb7f3474e44a6a9039eadb187ea9e, State: Initialized, Role: FOLLOWER
I20260808 06:05:09.895933 14347 consensus_queue.cc:260] T 376c349bf9a74fb28ea7a0e46e2c7066 P 0ecfb7f3474e44a6a9039eadb187ea9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } }
I20260808 06:05:09.896188 14347 ts_tablet_manager.cc:1428] T 376c349bf9a74fb28ea7a0e46e2c7066 P 0ecfb7f3474e44a6a9039eadb187ea9e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 06:05:09.896878 14352 tablet_bootstrap.cc:492] T 376c349bf9a74fb28ea7a0e46e2c7066 P d784a288c65342a8a9b126e4ec8a618f: No bootstrap required, opened a new log
I20260808 06:05:09.896965 14352 ts_tablet_manager.cc:1397] T 376c349bf9a74fb28ea7a0e46e2c7066 P d784a288c65342a8a9b126e4ec8a618f: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260808 06:05:09.897436 14352 raft_consensus.cc:348] T 376c349bf9a74fb28ea7a0e46e2c7066 P d784a288c65342a8a9b126e4ec8a618f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } }
I20260808 06:05:09.897544 14352 raft_consensus.cc:374] T 376c349bf9a74fb28ea7a0e46e2c7066 P d784a288c65342a8a9b126e4ec8a618f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:05:09.897575 14352 raft_consensus.cc:729] T 376c349bf9a74fb28ea7a0e46e2c7066 P d784a288c65342a8a9b126e4ec8a618f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d784a288c65342a8a9b126e4ec8a618f, State: Initialized, Role: FOLLOWER
I20260808 06:05:09.897686 14352 consensus_queue.cc:260] T 376c349bf9a74fb28ea7a0e46e2c7066 P d784a288c65342a8a9b126e4ec8a618f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } }
I20260808 06:05:09.897960 14352 ts_tablet_manager.cc:1428] T 376c349bf9a74fb28ea7a0e46e2c7066 P d784a288c65342a8a9b126e4ec8a618f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 06:05:09.900290 14350 tablet_bootstrap.cc:492] T 376c349bf9a74fb28ea7a0e46e2c7066 P d2678bd25fe44d98b1515f2188a9d777: No bootstrap required, opened a new log
I20260808 06:05:09.900375 14350 ts_tablet_manager.cc:1397] T 376c349bf9a74fb28ea7a0e46e2c7066 P d2678bd25fe44d98b1515f2188a9d777: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260808 06:05:09.900823 14350 raft_consensus.cc:348] T 376c349bf9a74fb28ea7a0e46e2c7066 P d2678bd25fe44d98b1515f2188a9d777 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } }
I20260808 06:05:09.900918 14350 raft_consensus.cc:374] T 376c349bf9a74fb28ea7a0e46e2c7066 P d2678bd25fe44d98b1515f2188a9d777 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:05:09.900947 14350 raft_consensus.cc:729] T 376c349bf9a74fb28ea7a0e46e2c7066 P d2678bd25fe44d98b1515f2188a9d777 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2678bd25fe44d98b1515f2188a9d777, State: Initialized, Role: FOLLOWER
I20260808 06:05:09.901069 14350 consensus_queue.cc:260] T 376c349bf9a74fb28ea7a0e46e2c7066 P d2678bd25fe44d98b1515f2188a9d777 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } }
I20260808 06:05:09.901301 14350 ts_tablet_manager.cc:1428] T 376c349bf9a74fb28ea7a0e46e2c7066 P d2678bd25fe44d98b1515f2188a9d777: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 06:05:10.289973 14379 raft_consensus.cc:482] T 376c349bf9a74fb28ea7a0e46e2c7066 P 0ecfb7f3474e44a6a9039eadb187ea9e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:05:10.290125 14379 raft_consensus.cc:504] T 376c349bf9a74fb28ea7a0e46e2c7066 P 0ecfb7f3474e44a6a9039eadb187ea9e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } }
I20260808 06:05:10.290567 14379 leader_election.cc:288] T 376c349bf9a74fb28ea7a0e46e2c7066 P 0ecfb7f3474e44a6a9039eadb187ea9e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d784a288c65342a8a9b126e4ec8a618f (127.1.151.129:35179), d2678bd25fe44d98b1515f2188a9d777 (127.1.151.131:35921)
I20260808 06:05:10.291100 13779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "376c349bf9a74fb28ea7a0e46e2c7066" candidate_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d784a288c65342a8a9b126e4ec8a618f" is_pre_election: true
I20260808 06:05:10.291193 14236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "376c349bf9a74fb28ea7a0e46e2c7066" candidate_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2678bd25fe44d98b1515f2188a9d777" is_pre_election: true
I20260808 06:05:10.291282 13779 raft_consensus.cc:2454] T 376c349bf9a74fb28ea7a0e46e2c7066 P d784a288c65342a8a9b126e4ec8a618f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ecfb7f3474e44a6a9039eadb187ea9e in term 0.
I20260808 06:05:10.291360 14236 raft_consensus.cc:2454] T 376c349bf9a74fb28ea7a0e46e2c7066 P d2678bd25fe44d98b1515f2188a9d777 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ecfb7f3474e44a6a9039eadb187ea9e in term 0.
I20260808 06:05:10.291708 13897 leader_election.cc:302] T 376c349bf9a74fb28ea7a0e46e2c7066 P 0ecfb7f3474e44a6a9039eadb187ea9e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ecfb7f3474e44a6a9039eadb187ea9e, d784a288c65342a8a9b126e4ec8a618f; no voters:
I20260808 06:05:10.292035 14379 raft_consensus.cc:2781] T 376c349bf9a74fb28ea7a0e46e2c7066 P 0ecfb7f3474e44a6a9039eadb187ea9e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 06:05:10.292099 14379 raft_consensus.cc:482] T 376c349bf9a74fb28ea7a0e46e2c7066 P 0ecfb7f3474e44a6a9039eadb187ea9e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 06:05:10.292126 14379 raft_consensus.cc:3037] T 376c349bf9a74fb28ea7a0e46e2c7066 P 0ecfb7f3474e44a6a9039eadb187ea9e [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:05:10.293488 14379 raft_consensus.cc:504] T 376c349bf9a74fb28ea7a0e46e2c7066 P 0ecfb7f3474e44a6a9039eadb187ea9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } }
I20260808 06:05:10.293871 14379 leader_election.cc:288] T 376c349bf9a74fb28ea7a0e46e2c7066 P 0ecfb7f3474e44a6a9039eadb187ea9e [CANDIDATE]: Term 1 election: Requested vote from peers d784a288c65342a8a9b126e4ec8a618f (127.1.151.129:35179), d2678bd25fe44d98b1515f2188a9d777 (127.1.151.131:35921)
I20260808 06:05:10.294255 13779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "376c349bf9a74fb28ea7a0e46e2c7066" candidate_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d784a288c65342a8a9b126e4ec8a618f"
I20260808 06:05:10.294380 13779 raft_consensus.cc:3037] T 376c349bf9a74fb28ea7a0e46e2c7066 P d784a288c65342a8a9b126e4ec8a618f [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:05:10.295559 13779 raft_consensus.cc:2454] T 376c349bf9a74fb28ea7a0e46e2c7066 P d784a288c65342a8a9b126e4ec8a618f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ecfb7f3474e44a6a9039eadb187ea9e in term 1.
I20260808 06:05:10.295943 13897 leader_election.cc:302] T 376c349bf9a74fb28ea7a0e46e2c7066 P 0ecfb7f3474e44a6a9039eadb187ea9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ecfb7f3474e44a6a9039eadb187ea9e, d784a288c65342a8a9b126e4ec8a618f; no voters:
I20260808 06:05:10.300203 14236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "376c349bf9a74fb28ea7a0e46e2c7066" candidate_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2678bd25fe44d98b1515f2188a9d777"
I20260808 06:05:10.300343 14236 raft_consensus.cc:3037] T 376c349bf9a74fb28ea7a0e46e2c7066 P d2678bd25fe44d98b1515f2188a9d777 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:05:10.301306 14236 raft_consensus.cc:2454] T 376c349bf9a74fb28ea7a0e46e2c7066 P d2678bd25fe44d98b1515f2188a9d777 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ecfb7f3474e44a6a9039eadb187ea9e in term 1.
I20260808 06:05:10.301509 14379 raft_consensus.cc:2781] T 376c349bf9a74fb28ea7a0e46e2c7066 P 0ecfb7f3474e44a6a9039eadb187ea9e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:05:10.301709 14379 raft_consensus.cc:686] T 376c349bf9a74fb28ea7a0e46e2c7066 P 0ecfb7f3474e44a6a9039eadb187ea9e [term 1 LEADER]: Becoming Leader. State: Replica: 0ecfb7f3474e44a6a9039eadb187ea9e, State: Running, Role: LEADER
I20260808 06:05:10.301858 14379 consensus_queue.cc:237] T 376c349bf9a74fb28ea7a0e46e2c7066 P 0ecfb7f3474e44a6a9039eadb187ea9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } }
I20260808 06:05:10.303539 13535 catalog_manager.cc:5495] T 376c349bf9a74fb28ea7a0e46e2c7066 P 0ecfb7f3474e44a6a9039eadb187ea9e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ecfb7f3474e44a6a9039eadb187ea9e (127.1.151.130). New cstate: current_term: 1 leader_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } health_report { overall_health: UNKNOWN } } }
I20260808 06:05:10.353442 14474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:10.350501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38734 (local address 127.1.151.130:44373)
0808 06:05:10.350769 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:10.350773 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:10.350786 (+ 13us) server_negotiation.cc:407] Connection header received
0808 06:05:10.350832 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:10.350834 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:10.350876 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:10.350945 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:05:10.351211 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:10.351744 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:10.352559 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:10.352786 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:10.352863 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:10.353058 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:05:10.353128 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:05:10.353176 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:10.353241 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 06:05:10.353287 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":111,"threads_started":1}
I20260808 06:05:10.355500 14236 raft_consensus.cc:1261] T 376c349bf9a74fb28ea7a0e46e2c7066 P d2678bd25fe44d98b1515f2188a9d777 [term 1 FOLLOWER]: Refusing update from remote peer 0ecfb7f3474e44a6a9039eadb187ea9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:05:10.355551 13779 raft_consensus.cc:1261] T 376c349bf9a74fb28ea7a0e46e2c7066 P d784a288c65342a8a9b126e4ec8a618f [term 1 FOLLOWER]: Refusing update from remote peer 0ecfb7f3474e44a6a9039eadb187ea9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:05:10.355929 14379 consensus_queue.cc:1035] T 376c349bf9a74fb28ea7a0e46e2c7066 P 0ecfb7f3474e44a6a9039eadb187ea9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:05:10.356093 14471 consensus_queue.cc:1035] T 376c349bf9a74fb28ea7a0e46e2c7066 P 0ecfb7f3474e44a6a9039eadb187ea9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:05:10.363677 13535 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:05:10.363796 13535 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:05:10.365506 13535 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f5f0153a8bbb4a819b1bb2eebff8ca4f: Sending DeleteTablet for 3 replicas of tablet 376c349bf9a74fb28ea7a0e46e2c7066
I20260808 06:05:10.366068 13973 tablet_service.cc:1514] Processing DeleteTablet for tablet 376c349bf9a74fb28ea7a0e46e2c7066 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:05:10 UTC) from {username='oryx', principal='oryx/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:38722
I20260808 06:05:10.366094 14206 tablet_service.cc:1514] Processing DeleteTablet for tablet 376c349bf9a74fb28ea7a0e46e2c7066 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:05:10 UTC) from {username='oryx', principal='oryx/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:47214
I20260808 06:05:10.366103 13750 tablet_service.cc:1514] Processing DeleteTablet for tablet 376c349bf9a74fb28ea7a0e46e2c7066 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:05:10 UTC) from {username='oryx', principal='oryx/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:41262
I20260808 06:05:10.366431 14487 tablet_replica.cc:330] T 376c349bf9a74fb28ea7a0e46e2c7066 P d2678bd25fe44d98b1515f2188a9d777: stopping tablet replica
I20260808 06:05:10.366552 14487 raft_consensus.cc:2229] T 376c349bf9a74fb28ea7a0e46e2c7066 P d2678bd25fe44d98b1515f2188a9d777 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:05:10.366683 14487 raft_consensus.cc:2258] T 376c349bf9a74fb28ea7a0e46e2c7066 P d2678bd25fe44d98b1515f2188a9d777 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:05:10.366689 13535 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47770:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 06:05:10.366904 14488 tablet_replica.cc:330] T 376c349bf9a74fb28ea7a0e46e2c7066 P 0ecfb7f3474e44a6a9039eadb187ea9e: stopping tablet replica
I20260808 06:05:10.367024 14488 raft_consensus.cc:2229] T 376c349bf9a74fb28ea7a0e46e2c7066 P 0ecfb7f3474e44a6a9039eadb187ea9e [term 1 LEADER]: Raft consensus shutting down.
W20260808 06:05:10.367064 13535 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 06:05:10.367201 14488 raft_consensus.cc:2258] T 376c349bf9a74fb28ea7a0e46e2c7066 P 0ecfb7f3474e44a6a9039eadb187ea9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:05:10.367419 14491 tablet_replica.cc:330] T 376c349bf9a74fb28ea7a0e46e2c7066 P d784a288c65342a8a9b126e4ec8a618f: stopping tablet replica
I20260808 06:05:10.367429 14487 ts_tablet_manager.cc:1905] T 376c349bf9a74fb28ea7a0e46e2c7066 P d2678bd25fe44d98b1515f2188a9d777: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:05:10.367534 14491 raft_consensus.cc:2229] T 376c349bf9a74fb28ea7a0e46e2c7066 P d784a288c65342a8a9b126e4ec8a618f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:05:10.367663 14491 raft_consensus.cc:2258] T 376c349bf9a74fb28ea7a0e46e2c7066 P d784a288c65342a8a9b126e4ec8a618f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:05:10.368207 14488 ts_tablet_manager.cc:1905] T 376c349bf9a74fb28ea7a0e46e2c7066 P 0ecfb7f3474e44a6a9039eadb187ea9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:05:10.368736 14491 ts_tablet_manager.cc:1905] T 376c349bf9a74fb28ea7a0e46e2c7066 P d784a288c65342a8a9b126e4ec8a618f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:05:10.370007 14487 ts_tablet_manager.cc:1918] T 376c349bf9a74fb28ea7a0e46e2c7066 P d2678bd25fe44d98b1515f2188a9d777: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:05:10.370095 14487 log.cc:1196] T 376c349bf9a74fb28ea7a0e46e2c7066 P d2678bd25fe44d98b1515f2188a9d777: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-2/wal/wals/376c349bf9a74fb28ea7a0e46e2c7066
I20260808 06:05:10.370415 14487 ts_tablet_manager.cc:1939] T 376c349bf9a74fb28ea7a0e46e2c7066 P d2678bd25fe44d98b1515f2188a9d777: Deleting consensus metadata
I20260808 06:05:10.371104 13518 catalog_manager.cc:4841] TS d2678bd25fe44d98b1515f2188a9d777 (127.1.151.131:35921): tablet 376c349bf9a74fb28ea7a0e46e2c7066 (table test-table [id=4ec32127323741ee866356a7e943cd4d]) successfully deleted
I20260808 06:05:10.371572 14491 ts_tablet_manager.cc:1918] T 376c349bf9a74fb28ea7a0e46e2c7066 P d784a288c65342a8a9b126e4ec8a618f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:05:10.371652 14491 log.cc:1196] T 376c349bf9a74fb28ea7a0e46e2c7066 P d784a288c65342a8a9b126e4ec8a618f: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-0/wal/wals/376c349bf9a74fb28ea7a0e46e2c7066
I20260808 06:05:10.371908 14491 ts_tablet_manager.cc:1939] T 376c349bf9a74fb28ea7a0e46e2c7066 P d784a288c65342a8a9b126e4ec8a618f: Deleting consensus metadata
I20260808 06:05:10.372015 13750 tablet_service.cc:1467] Processing CreateTablet for tablet 01e6ccca83ac46a082facb17bad5c5ec (DEFAULT_TABLE table=test-table [id=5f46d2ed8b434fed9e9bf60bb3b6e772]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:05:10.372030 13973 tablet_service.cc:1467] Processing CreateTablet for tablet 01e6ccca83ac46a082facb17bad5c5ec (DEFAULT_TABLE table=test-table [id=5f46d2ed8b434fed9e9bf60bb3b6e772]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:05:10.372260 13973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01e6ccca83ac46a082facb17bad5c5ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:05:10.372282 13750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01e6ccca83ac46a082facb17bad5c5ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:05:10.372984 14488 ts_tablet_manager.cc:1918] T 376c349bf9a74fb28ea7a0e46e2c7066 P 0ecfb7f3474e44a6a9039eadb187ea9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:05:10.373049 14488 log.cc:1196] T 376c349bf9a74fb28ea7a0e46e2c7066 P 0ecfb7f3474e44a6a9039eadb187ea9e: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-1/wal/wals/376c349bf9a74fb28ea7a0e46e2c7066
I20260808 06:05:10.373281 14488 ts_tablet_manager.cc:1939] T 376c349bf9a74fb28ea7a0e46e2c7066 P 0ecfb7f3474e44a6a9039eadb187ea9e: Deleting consensus metadata
I20260808 06:05:10.373941 14347 tablet_bootstrap.cc:492] T 01e6ccca83ac46a082facb17bad5c5ec P 0ecfb7f3474e44a6a9039eadb187ea9e: Bootstrap starting.
I20260808 06:05:10.373941 13519 catalog_manager.cc:4841] TS 0ecfb7f3474e44a6a9039eadb187ea9e (127.1.151.130:44373): tablet 376c349bf9a74fb28ea7a0e46e2c7066 (table test-table [id=4ec32127323741ee866356a7e943cd4d]) successfully deleted
I20260808 06:05:10.374513 13518 catalog_manager.cc:4841] TS d784a288c65342a8a9b126e4ec8a618f (127.1.151.129:35179): tablet 376c349bf9a74fb28ea7a0e46e2c7066 (table test-table [id=4ec32127323741ee866356a7e943cd4d]) successfully deleted
I20260808 06:05:10.374774 14206 tablet_service.cc:1467] Processing CreateTablet for tablet 01e6ccca83ac46a082facb17bad5c5ec (DEFAULT_TABLE table=test-table [id=5f46d2ed8b434fed9e9bf60bb3b6e772]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:05:10.374835 14347 tablet_bootstrap.cc:654] T 01e6ccca83ac46a082facb17bad5c5ec P 0ecfb7f3474e44a6a9039eadb187ea9e: Neither blocks nor log segments found. Creating new log.
I20260808 06:05:10.374995 14206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01e6ccca83ac46a082facb17bad5c5ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:05:10.375371 14352 tablet_bootstrap.cc:492] T 01e6ccca83ac46a082facb17bad5c5ec P d784a288c65342a8a9b126e4ec8a618f: Bootstrap starting.
I20260808 06:05:10.376073 14352 tablet_bootstrap.cc:654] T 01e6ccca83ac46a082facb17bad5c5ec P d784a288c65342a8a9b126e4ec8a618f: Neither blocks nor log segments found. Creating new log.
I20260808 06:05:10.376752 14350 tablet_bootstrap.cc:492] T 01e6ccca83ac46a082facb17bad5c5ec P d2678bd25fe44d98b1515f2188a9d777: Bootstrap starting.
I20260808 06:05:10.377256 14352 tablet_bootstrap.cc:492] T 01e6ccca83ac46a082facb17bad5c5ec P d784a288c65342a8a9b126e4ec8a618f: No bootstrap required, opened a new log
I20260808 06:05:10.377336 14352 ts_tablet_manager.cc:1397] T 01e6ccca83ac46a082facb17bad5c5ec P d784a288c65342a8a9b126e4ec8a618f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 06:05:10.377526 14350 tablet_bootstrap.cc:654] T 01e6ccca83ac46a082facb17bad5c5ec P d2678bd25fe44d98b1515f2188a9d777: Neither blocks nor log segments found. Creating new log.
I20260808 06:05:10.377710 14352 raft_consensus.cc:348] T 01e6ccca83ac46a082facb17bad5c5ec P d784a288c65342a8a9b126e4ec8a618f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } }
I20260808 06:05:10.377801 14352 raft_consensus.cc:374] T 01e6ccca83ac46a082facb17bad5c5ec P d784a288c65342a8a9b126e4ec8a618f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:05:10.377830 14352 raft_consensus.cc:729] T 01e6ccca83ac46a082facb17bad5c5ec P d784a288c65342a8a9b126e4ec8a618f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d784a288c65342a8a9b126e4ec8a618f, State: Initialized, Role: FOLLOWER
I20260808 06:05:10.377966 14352 consensus_queue.cc:260] T 01e6ccca83ac46a082facb17bad5c5ec P d784a288c65342a8a9b126e4ec8a618f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } }
I20260808 06:05:10.378254 14352 ts_tablet_manager.cc:1428] T 01e6ccca83ac46a082facb17bad5c5ec P d784a288c65342a8a9b126e4ec8a618f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 06:05:10.378526 14350 tablet_bootstrap.cc:492] T 01e6ccca83ac46a082facb17bad5c5ec P d2678bd25fe44d98b1515f2188a9d777: No bootstrap required, opened a new log
I20260808 06:05:10.378594 14350 ts_tablet_manager.cc:1397] T 01e6ccca83ac46a082facb17bad5c5ec P d2678bd25fe44d98b1515f2188a9d777: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 06:05:10.378753 14347 tablet_bootstrap.cc:492] T 01e6ccca83ac46a082facb17bad5c5ec P 0ecfb7f3474e44a6a9039eadb187ea9e: No bootstrap required, opened a new log
I20260808 06:05:10.378830 14347 ts_tablet_manager.cc:1397] T 01e6ccca83ac46a082facb17bad5c5ec P 0ecfb7f3474e44a6a9039eadb187ea9e: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260808 06:05:10.378944 14350 raft_consensus.cc:348] T 01e6ccca83ac46a082facb17bad5c5ec P d2678bd25fe44d98b1515f2188a9d777 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } }
I20260808 06:05:10.379055 14350 raft_consensus.cc:374] T 01e6ccca83ac46a082facb17bad5c5ec P d2678bd25fe44d98b1515f2188a9d777 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:05:10.379086 14350 raft_consensus.cc:729] T 01e6ccca83ac46a082facb17bad5c5ec P d2678bd25fe44d98b1515f2188a9d777 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2678bd25fe44d98b1515f2188a9d777, State: Initialized, Role: FOLLOWER
I20260808 06:05:10.379184 14350 consensus_queue.cc:260] T 01e6ccca83ac46a082facb17bad5c5ec P d2678bd25fe44d98b1515f2188a9d777 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } }
I20260808 06:05:10.379196 14347 raft_consensus.cc:348] T 01e6ccca83ac46a082facb17bad5c5ec P 0ecfb7f3474e44a6a9039eadb187ea9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } }
I20260808 06:05:10.379302 14347 raft_consensus.cc:374] T 01e6ccca83ac46a082facb17bad5c5ec P 0ecfb7f3474e44a6a9039eadb187ea9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:05:10.379333 14347 raft_consensus.cc:729] T 01e6ccca83ac46a082facb17bad5c5ec P 0ecfb7f3474e44a6a9039eadb187ea9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ecfb7f3474e44a6a9039eadb187ea9e, State: Initialized, Role: FOLLOWER
I20260808 06:05:10.379416 14350 ts_tablet_manager.cc:1428] T 01e6ccca83ac46a082facb17bad5c5ec P d2678bd25fe44d98b1515f2188a9d777: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 06:05:10.379463 14347 consensus_queue.cc:260] T 01e6ccca83ac46a082facb17bad5c5ec P 0ecfb7f3474e44a6a9039eadb187ea9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } }
I20260808 06:05:10.379738 14347 ts_tablet_manager.cc:1428] T 01e6ccca83ac46a082facb17bad5c5ec P 0ecfb7f3474e44a6a9039eadb187ea9e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 06:05:10.436468 14443 raft_consensus.cc:482] T 01e6ccca83ac46a082facb17bad5c5ec P d784a288c65342a8a9b126e4ec8a618f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:05:10.436602 14443 raft_consensus.cc:504] T 01e6ccca83ac46a082facb17bad5c5ec P d784a288c65342a8a9b126e4ec8a618f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } }
I20260808 06:05:10.436982 14443 leader_election.cc:288] T 01e6ccca83ac46a082facb17bad5c5ec P d784a288c65342a8a9b126e4ec8a618f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ecfb7f3474e44a6a9039eadb187ea9e (127.1.151.130:44373), d2678bd25fe44d98b1515f2188a9d777 (127.1.151.131:35921)
I20260808 06:05:10.437359 14004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01e6ccca83ac46a082facb17bad5c5ec" candidate_uuid: "d784a288c65342a8a9b126e4ec8a618f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" is_pre_election: true
I20260808 06:05:10.437430 14236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01e6ccca83ac46a082facb17bad5c5ec" candidate_uuid: "d784a288c65342a8a9b126e4ec8a618f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2678bd25fe44d98b1515f2188a9d777" is_pre_election: true
I20260808 06:05:10.437502 14004 raft_consensus.cc:2454] T 01e6ccca83ac46a082facb17bad5c5ec P 0ecfb7f3474e44a6a9039eadb187ea9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d784a288c65342a8a9b126e4ec8a618f in term 0.
I20260808 06:05:10.437587 14236 raft_consensus.cc:2454] T 01e6ccca83ac46a082facb17bad5c5ec P d2678bd25fe44d98b1515f2188a9d777 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d784a288c65342a8a9b126e4ec8a618f in term 0.
I20260808 06:05:10.437794 13665 leader_election.cc:302] T 01e6ccca83ac46a082facb17bad5c5ec P d784a288c65342a8a9b126e4ec8a618f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ecfb7f3474e44a6a9039eadb187ea9e, d784a288c65342a8a9b126e4ec8a618f; no voters:
I20260808 06:05:10.438023 14443 raft_consensus.cc:2781] T 01e6ccca83ac46a082facb17bad5c5ec P d784a288c65342a8a9b126e4ec8a618f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 06:05:10.438071 14443 raft_consensus.cc:482] T 01e6ccca83ac46a082facb17bad5c5ec P d784a288c65342a8a9b126e4ec8a618f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 06:05:10.438099 14443 raft_consensus.cc:3037] T 01e6ccca83ac46a082facb17bad5c5ec P d784a288c65342a8a9b126e4ec8a618f [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:05:10.439103 14443 raft_consensus.cc:504] T 01e6ccca83ac46a082facb17bad5c5ec P d784a288c65342a8a9b126e4ec8a618f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } }
I20260808 06:05:10.439401 14443 leader_election.cc:288] T 01e6ccca83ac46a082facb17bad5c5ec P d784a288c65342a8a9b126e4ec8a618f [CANDIDATE]: Term 1 election: Requested vote from peers 0ecfb7f3474e44a6a9039eadb187ea9e (127.1.151.130:44373), d2678bd25fe44d98b1515f2188a9d777 (127.1.151.131:35921)
I20260808 06:05:10.439733 14004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01e6ccca83ac46a082facb17bad5c5ec" candidate_uuid: "d784a288c65342a8a9b126e4ec8a618f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e"
I20260808 06:05:10.439755 14236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01e6ccca83ac46a082facb17bad5c5ec" candidate_uuid: "d784a288c65342a8a9b126e4ec8a618f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2678bd25fe44d98b1515f2188a9d777"
I20260808 06:05:10.439844 14004 raft_consensus.cc:3037] T 01e6ccca83ac46a082facb17bad5c5ec P 0ecfb7f3474e44a6a9039eadb187ea9e [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:05:10.439872 14236 raft_consensus.cc:3037] T 01e6ccca83ac46a082facb17bad5c5ec P d2678bd25fe44d98b1515f2188a9d777 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:05:10.440735 14236 raft_consensus.cc:2454] T 01e6ccca83ac46a082facb17bad5c5ec P d2678bd25fe44d98b1515f2188a9d777 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d784a288c65342a8a9b126e4ec8a618f in term 1.
I20260808 06:05:10.440789 14004 raft_consensus.cc:2454] T 01e6ccca83ac46a082facb17bad5c5ec P 0ecfb7f3474e44a6a9039eadb187ea9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d784a288c65342a8a9b126e4ec8a618f in term 1.
I20260808 06:05:10.440996 13663 leader_election.cc:302] T 01e6ccca83ac46a082facb17bad5c5ec P d784a288c65342a8a9b126e4ec8a618f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2678bd25fe44d98b1515f2188a9d777, d784a288c65342a8a9b126e4ec8a618f; no voters:
I20260808 06:05:10.441155 14443 raft_consensus.cc:2781] T 01e6ccca83ac46a082facb17bad5c5ec P d784a288c65342a8a9b126e4ec8a618f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:05:10.441356 14443 raft_consensus.cc:686] T 01e6ccca83ac46a082facb17bad5c5ec P d784a288c65342a8a9b126e4ec8a618f [term 1 LEADER]: Becoming Leader. State: Replica: d784a288c65342a8a9b126e4ec8a618f, State: Running, Role: LEADER
I20260808 06:05:10.441495 14443 consensus_queue.cc:237] T 01e6ccca83ac46a082facb17bad5c5ec P d784a288c65342a8a9b126e4ec8a618f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } }
I20260808 06:05:10.443063 13535 catalog_manager.cc:5495] T 01e6ccca83ac46a082facb17bad5c5ec P d784a288c65342a8a9b126e4ec8a618f reported cstate change: term changed from 0 to 1, leader changed from <none> to d784a288c65342a8a9b126e4ec8a618f (127.1.151.129). New cstate: current_term: 1 leader_uuid: "d784a288c65342a8a9b126e4ec8a618f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d784a288c65342a8a9b126e4ec8a618f" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35179 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 } health_report { overall_health: UNKNOWN } } }
I20260808 06:05:10.462513 14501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:10.459663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41294 (local address 127.1.151.129:35179)
0808 06:05:10.459896 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:10.459902 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:10.459914 (+ 12us) server_negotiation.cc:407] Connection header received
0808 06:05:10.459953 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:10.459955 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:10.459995 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:10.460062 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:05:10.460286 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:10.460867 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:10.461653 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:10.461876 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:10.461951 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:10.462125 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:05:10.462233 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:05:10.462285 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:10.462357 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 06:05:10.462399 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":98,"threads_started":1}
I20260808 06:05:10.468210 14236 raft_consensus.cc:1261] T 01e6ccca83ac46a082facb17bad5c5ec P d2678bd25fe44d98b1515f2188a9d777 [term 1 FOLLOWER]: Refusing update from remote peer d784a288c65342a8a9b126e4ec8a618f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:05:10.468545 14004 raft_consensus.cc:1261] T 01e6ccca83ac46a082facb17bad5c5ec P 0ecfb7f3474e44a6a9039eadb187ea9e [term 1 FOLLOWER]: Refusing update from remote peer d784a288c65342a8a9b126e4ec8a618f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:05:10.468639 14495 consensus_queue.cc:1035] T 01e6ccca83ac46a082facb17bad5c5ec P d784a288c65342a8a9b126e4ec8a618f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2678bd25fe44d98b1515f2188a9d777" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:05:10.468983 14495 consensus_queue.cc:1035] T 01e6ccca83ac46a082facb17bad5c5ec P d784a288c65342a8a9b126e4ec8a618f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ecfb7f3474e44a6a9039eadb187ea9e" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:05:10.497859 13534 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:05:10.498003 13534 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:05:10.499897 13534 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f5f0153a8bbb4a819b1bb2eebff8ca4f: Sending DeleteTablet for 3 replicas of tablet 01e6ccca83ac46a082facb17bad5c5ec
I20260808 06:05:10.500379 13750 tablet_service.cc:1514] Processing DeleteTablet for tablet 01e6ccca83ac46a082facb17bad5c5ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:05:10 UTC) from {username='oryx', principal='oryx/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:41262
I20260808 06:05:10.500438 14206 tablet_service.cc:1514] Processing DeleteTablet for tablet 01e6ccca83ac46a082facb17bad5c5ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:05:10 UTC) from {username='oryx', principal='oryx/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:47214
I20260808 06:05:10.500447 13973 tablet_service.cc:1514] Processing DeleteTablet for tablet 01e6ccca83ac46a082facb17bad5c5ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:05:10 UTC) from {username='oryx', principal='oryx/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:38722
I20260808 06:05:10.500571 14491 tablet_replica.cc:330] T 01e6ccca83ac46a082facb17bad5c5ec P d784a288c65342a8a9b126e4ec8a618f: stopping tablet replica
I20260808 06:05:10.500578 14487 tablet_replica.cc:330] T 01e6ccca83ac46a082facb17bad5c5ec P d2678bd25fe44d98b1515f2188a9d777: stopping tablet replica
I20260808 06:05:10.500633 14488 tablet_replica.cc:330] T 01e6ccca83ac46a082facb17bad5c5ec P 0ecfb7f3474e44a6a9039eadb187ea9e: stopping tablet replica
I20260808 06:05:10.500672 14491 raft_consensus.cc:2229] T 01e6ccca83ac46a082facb17bad5c5ec P d784a288c65342a8a9b126e4ec8a618f [term 1 LEADER]: Raft consensus shutting down.
I20260808 06:05:10.500672 14487 raft_consensus.cc:2229] T 01e6ccca83ac46a082facb17bad5c5ec P d2678bd25fe44d98b1515f2188a9d777 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:05:10.500734 14488 raft_consensus.cc:2229] T 01e6ccca83ac46a082facb17bad5c5ec P 0ecfb7f3474e44a6a9039eadb187ea9e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:05:10.500764 14487 raft_consensus.cc:2258] T 01e6ccca83ac46a082facb17bad5c5ec P d2678bd25fe44d98b1515f2188a9d777 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:05:10.500814 14488 raft_consensus.cc:2258] T 01e6ccca83ac46a082facb17bad5c5ec P 0ecfb7f3474e44a6a9039eadb187ea9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:05:10.500836 14491 raft_consensus.cc:2258] T 01e6ccca83ac46a082facb17bad5c5ec P d784a288c65342a8a9b126e4ec8a618f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:05:10.501441 14488 ts_tablet_manager.cc:1905] T 01e6ccca83ac46a082facb17bad5c5ec P 0ecfb7f3474e44a6a9039eadb187ea9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:05:10.501573 14491 ts_tablet_manager.cc:1905] T 01e6ccca83ac46a082facb17bad5c5ec P d784a288c65342a8a9b126e4ec8a618f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:05:10.501574 14487 ts_tablet_manager.cc:1905] T 01e6ccca83ac46a082facb17bad5c5ec P d2678bd25fe44d98b1515f2188a9d777: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:05:10.501788 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 13645
I20260808 06:05:10.503895 14488 ts_tablet_manager.cc:1918] T 01e6ccca83ac46a082facb17bad5c5ec P 0ecfb7f3474e44a6a9039eadb187ea9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 06:05:10.503957 14488 log.cc:1196] T 01e6ccca83ac46a082facb17bad5c5ec P 0ecfb7f3474e44a6a9039eadb187ea9e: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-1/wal/wals/01e6ccca83ac46a082facb17bad5c5ec
I20260808 06:05:10.504153 14488 ts_tablet_manager.cc:1939] T 01e6ccca83ac46a082facb17bad5c5ec P 0ecfb7f3474e44a6a9039eadb187ea9e: Deleting consensus metadata
I20260808 06:05:10.504266 14487 ts_tablet_manager.cc:1918] T 01e6ccca83ac46a082facb17bad5c5ec P d2678bd25fe44d98b1515f2188a9d777: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 06:05:10.504376 14487 log.cc:1196] T 01e6ccca83ac46a082facb17bad5c5ec P d2678bd25fe44d98b1515f2188a9d777: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786169070948234-1630-0/minicluster-data/ts-2/wal/wals/01e6ccca83ac46a082facb17bad5c5ec
I20260808 06:05:10.504676 14487 ts_tablet_manager.cc:1939] T 01e6ccca83ac46a082facb17bad5c5ec P d2678bd25fe44d98b1515f2188a9d777: Deleting consensus metadata
I20260808 06:05:10.505808 13518 catalog_manager.cc:4841] TS d2678bd25fe44d98b1515f2188a9d777 (127.1.151.131:35921): tablet 01e6ccca83ac46a082facb17bad5c5ec (table test-table [id=5f46d2ed8b434fed9e9bf60bb3b6e772]) successfully deleted
I20260808 06:05:10.513281 13519 catalog_manager.cc:4841] TS 0ecfb7f3474e44a6a9039eadb187ea9e (127.1.151.130:44373): tablet 01e6ccca83ac46a082facb17bad5c5ec (table test-table [id=5f46d2ed8b434fed9e9bf60bb3b6e772]) successfully deleted
W20260808 06:05:10.521064 13518 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.151.129:35179 (error 108)
I20260808 06:05:10.521313 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 13868
I20260808 06:05:10.525632 14516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:10.522196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:35179 (local address 127.0.0.1:41300)
0808 06:05:10.525436 (+ 3240us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:10.525514 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.129:35179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3157,"thread_start_us":1069,"threads_started":1}
W20260808 06:05:10.526407 13518 catalog_manager.cc:4584] TS d784a288c65342a8a9b126e4ec8a618f (127.1.151.129:35179): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 01e6ccca83ac46a082facb17bad5c5ec: Network error: Client connection negotiation failed: client connection to 127.1.151.129:35179: connect: Connection refused (error 111)
I20260808 06:05:10.535435 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 14105
I20260808 06:05:10.549160 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 13496
2026-08-08T06:05:10Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3745 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:05:10 dist-test-slave-zj8q krb5kdc[14521](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:05:10 dist-test-slave-zj8q krb5kdc[14521](info): set up 2 sockets
Aug 08 06:05:10 dist-test-slave-zj8q krb5kdc[14521](info): commencing operation
krb5kdc: starting...
W20260808 06:05:11.708210 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:05:11 dist-test-slave-zj8q krb5kdc[14521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169111, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:05:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:05:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.151.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
I20260808 06:05:11.829434 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:36177
--webserver_interface=127.1.151.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:40393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:36177
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:05:11.938680 14548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:11.938963 14548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:11.939019 14548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:11.942996 14548 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:05:11.943049 14548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:11.943075 14548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:11.943095 14548 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:05:11.943117 14548 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:05:11.948553 14548 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:40393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:36177
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:36177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:05:11.949538 14548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:11.950537 14548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:11.958632 14555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:11.958662 14554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:11.958829 14548 server_base.cc:1034] running on GCE node
W20260808 06:05:11.958662 14557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:11.959372 14548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:11.960407 14548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:11.961563 14548 hybrid_clock.cc:648] HybridClock initialized: now 1786169111961546 us; error 29 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T06:05:11Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1419 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:05:12 dist-test-slave-zj8q krb5kdc[14564](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:05:12 dist-test-slave-zj8q krb5kdc[14564](info): set up 2 sockets
Aug 08 06:05:12 dist-test-slave-zj8q krb5kdc[14564](info): commencing operation
krb5kdc: starting...
W20260808 06:05:13.125782 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:05:13 dist-test-slave-zj8q krb5kdc[14564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:05:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:05:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.151.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
I20260808 06:05:13.245251 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:34421
--webserver_interface=127.1.151.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:39549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:34421
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:05:13.356313 14582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:13.356573 14582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:13.356624 14582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:13.360585 14582 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:05:13.360643 14582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:13.360670 14582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:13.360690 14582 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:05:13.360711 14582 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:05:13.365888 14582 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:39549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:34421
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:34421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:05:13.366891 14582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:13.367867 14582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:13.375890 14589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:13.375900 14590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:13.375934 14582 server_base.cc:1034] running on GCE node
W20260808 06:05:13.375906 14592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:13.376605 14582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:13.377655 14582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:13.378813 14582 hybrid_clock.cc:648] HybridClock initialized: now 1786169113378791 us; error 34 us; skew 500 ppm
Aug 08 06:05:13 dist-test-slave-zj8q krb5kdc[14564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169113, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:13.385818 14582 init.cc:377] Logged in from keytab as kudu/127.1.151.190@KRBTEST.COM (short username kudu)
I20260808 06:05:13.386763 14582 webserver.cc:460] Webserver started at http://127.1.151.190:32905/ using document root <none> and password file <none>
I20260808 06:05:13.387337 14582 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:13.387406 14582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:13.387606 14582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:05:13.389622 14582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "264f566d98374115870725477fb4c5da"
format_stamp: "Formatted at 2026-08-08 06:05:13 on dist-test-slave-zj8q"
server_key: "0229edc6789e2e860b8d52e35a75bd6c"
server_key_iv: "83015728d9275d62193d2aed008a2877"
server_key_version: "encryptionkey@0"
I20260808 06:05:13.390220 14582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "264f566d98374115870725477fb4c5da"
format_stamp: "Formatted at 2026-08-08 06:05:13 on dist-test-slave-zj8q"
server_key: "0229edc6789e2e860b8d52e35a75bd6c"
server_key_iv: "83015728d9275d62193d2aed008a2877"
server_key_version: "encryptionkey@0"
I20260808 06:05:13.393869 14582 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 06:05:13.396261 14599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:13.397397 14582 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:05:13.397513 14582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "264f566d98374115870725477fb4c5da"
format_stamp: "Formatted at 2026-08-08 06:05:13 on dist-test-slave-zj8q"
server_key: "0229edc6789e2e860b8d52e35a75bd6c"
server_key_iv: "83015728d9275d62193d2aed008a2877"
server_key_version: "encryptionkey@0"
I20260808 06:05:13.397604 14582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:13.413941 14582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:13.416996 14582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:05:13.417179 14582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:13.423921 14582 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:34421
I20260808 06:05:13.423972 14683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:34421 every 8 connection(s)
I20260808 06:05:13.424829 14582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:05:13.427597 14684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:05:13.431195 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 14582
I20260808 06:05:13.431304 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:05:13.431562 1630 external_mini_cluster.cc:1428] Setting key 2803c7ec52b404ac21a778c9705f9746
I20260808 06:05:13.433173 14684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da: Bootstrap starting.
I20260808 06:05:13.435832 14684 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da: Neither blocks nor log segments found. Creating new log.
I20260808 06:05:13.436703 14684 log.cc:824] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da: Log is configured to *not* fsync() on all Append() calls
I20260808 06:05:13.438758 14684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da: No bootstrap required, opened a new log
Aug 08 06:05:13 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:05:13.442613 14684 raft_consensus.cc:348] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "264f566d98374115870725477fb4c5da" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34421 } }
I20260808 06:05:13.442826 14684 raft_consensus.cc:374] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:05:13.442864 14684 raft_consensus.cc:729] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 264f566d98374115870725477fb4c5da, State: Initialized, Role: FOLLOWER
I20260808 06:05:13.443269 14684 consensus_queue.cc:260] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "264f566d98374115870725477fb4c5da" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34421 } }
I20260808 06:05:13.443387 14684 raft_consensus.cc:388] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:05:13.443441 14684 raft_consensus.cc:482] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:05:13.443533 14684 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:05:13.444566 14684 raft_consensus.cc:504] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "264f566d98374115870725477fb4c5da" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34421 } }
I20260808 06:05:13.444895 14684 leader_election.cc:302] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 264f566d98374115870725477fb4c5da; no voters:
I20260808 06:05:13.445181 14684 leader_election.cc:288] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:05:13.445292 14691 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:05:13.445466 14691 raft_consensus.cc:686] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da [term 1 LEADER]: Becoming Leader. State: Replica: 264f566d98374115870725477fb4c5da, State: Running, Role: LEADER
I20260808 06:05:13.445827 14691 consensus_queue.cc:237] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "264f566d98374115870725477fb4c5da" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34421 } }
I20260808 06:05:13.446607 14684 sys_catalog.cc:564] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:05:13.446777 14689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:13.432974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55910 (local address 127.1.151.190:34421)
0808 06:05:13.433392 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:13.433412 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:13.433454 (+ 42us) server_negotiation.cc:407] Connection header received
0808 06:05:13.434304 (+ 850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:13.434330 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:13.434753 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:13.435096 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:13.435743 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:13.437207 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:13.438115 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:13.438824 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:13.438978 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:13.441747 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:13.441752 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:13.441766 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:13.441793 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:13.443611 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:13.444127 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:13.444132 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:13.444137 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:13.444254 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:13.444551 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:13.444552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:13.444554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:13.444989 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:13.445178 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:13.445649 (+ 471us) server_negotiation.cc:299] Negotiation successful
0808 06:05:13.445957 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":104,"threads_started":1}
I20260808 06:05:13.447104 14691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "264f566d98374115870725477fb4c5da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "264f566d98374115870725477fb4c5da" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34421 } } }
I20260808 06:05:13.447234 14691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da [sys.catalog]: This master's current role is: LEADER
I20260808 06:05:13.447431 14691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 264f566d98374115870725477fb4c5da. Latest consensus state: current_term: 1 leader_uuid: "264f566d98374115870725477fb4c5da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "264f566d98374115870725477fb4c5da" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34421 } } }
I20260808 06:05:13.447520 14691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da [sys.catalog]: This master's current role is: LEADER
I20260808 06:05:13.447670 14697 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:05:13.450860 14697 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:05:13.456233 14697 catalog_manager.cc:1347] Generated new cluster ID: f2fc48c20d2b4612bc2bae28c881b706
I20260808 06:05:13.456296 14697 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:05:13.468140 14697 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:05:13.469074 14697 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:05:13.473439 14697 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da: Generated new TSK 0
I20260808 06:05:13.474022 14697 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.151.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab.
Entry for principal kudu/127.1.151.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab.
I20260808 06:05:13.506649 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.129:0
--local_ip_for_outbound_sockets=127.1.151.129
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:34421
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:39549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:05:13.627224 14722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:13.627467 14722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:13.627512 14722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:13.631454 14722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:13.631503 14722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:13.631572 14722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.129
I20260808 06:05:13.636693 14722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:39549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:34421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.129
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:05:13.637658 14722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:13.638659 14722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:13.647928 14734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:13.647960 14729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:13.647957 14731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:13.648207 14722 server_base.cc:1034] running on GCE node
I20260808 06:05:13.648592 14722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:13.649219 14722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:13.650363 14722 hybrid_clock.cc:648] HybridClock initialized: now 1786169113650340 us; error 41 us; skew 500 ppm
Aug 08 06:05:13 dist-test-slave-zj8q krb5kdc[14564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169113, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:13.652926 14722 init.cc:377] Logged in from keytab as kudu/127.1.151.129@KRBTEST.COM (short username kudu)
I20260808 06:05:13.653718 14722 webserver.cc:460] Webserver started at http://127.1.151.129:44037/ using document root <none> and password file <none>
I20260808 06:05:13.654273 14722 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:13.654342 14722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:13.654534 14722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:05:13.656479 14722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-0/data/instance:
uuid: "1e365f2563f941eb8eff0714bd52ee46"
format_stamp: "Formatted at 2026-08-08 06:05:13 on dist-test-slave-zj8q"
server_key: "8090adea8a05640808c2209f01707734"
server_key_iv: "77a36c5551b84e23934a0f3755cfa44a"
server_key_version: "encryptionkey@0"
I20260808 06:05:13.657011 14722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance:
uuid: "1e365f2563f941eb8eff0714bd52ee46"
format_stamp: "Formatted at 2026-08-08 06:05:13 on dist-test-slave-zj8q"
server_key: "8090adea8a05640808c2209f01707734"
server_key_iv: "77a36c5551b84e23934a0f3755cfa44a"
server_key_version: "encryptionkey@0"
I20260808 06:05:13.660602 14722 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260808 06:05:13.662885 14747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:13.664050 14722 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:05:13.664175 14722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-0/wal
uuid: "1e365f2563f941eb8eff0714bd52ee46"
format_stamp: "Formatted at 2026-08-08 06:05:13 on dist-test-slave-zj8q"
server_key: "8090adea8a05640808c2209f01707734"
server_key_iv: "77a36c5551b84e23934a0f3755cfa44a"
server_key_version: "encryptionkey@0"
I20260808 06:05:13.664264 14722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:13.679087 14722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:13.681674 14722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:05:13.681836 14722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:13.682427 14722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:05:13.683219 14722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:05:13.683264 14722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:13.683316 14722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:05:13.683341 14722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:13.691465 14722 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.129:40433
I20260808 06:05:13.691497 14931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.129:40433 every 8 connection(s)
I20260808 06:05:13.692296 14722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
I20260808 06:05:13.693598 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 14722
I20260808 06:05:13.693735 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance
I20260808 06:05:13.694015 1630 external_mini_cluster.cc:1428] Setting key aaba87c0a02f4e2222e80ab52b5a5d1e
Aug 08 06:05:13 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169113, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.151.130@KRBTEST.COM; defaulting to no policy
I20260808 06:05:13.706974 14689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:13.694183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:43801 (local address 127.1.151.190:34421)
0808 06:05:13.694330 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:13.694334 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:13.695610 (+ 1276us) server_negotiation.cc:407] Connection header received
0808 06:05:13.696487 (+ 877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:13.696489 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:13.696536 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:13.696670 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:13.698121 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:13.698948 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:13.700491 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:13.700828 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:13.700939 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:13.703730 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:13.703733 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:13.703735 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:13.703760 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:13.705036 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:13.705477 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:13.705479 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:13.705480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:13.705559 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:13.705910 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:13.705915 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:13.705916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:13.706140 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:13.706257 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:13.706712 (+ 455us) server_negotiation.cc:299] Negotiation successful
0808 06:05:13.706813 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.130@KRBTEST.COM" created.
I20260808 06:05:13.707746 14936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:13.694590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.129:43801)
0808 06:05:13.695247 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:13.695297 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:13.696244 (+ 947us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:13.696865 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:13.696874 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:13.697340 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:13.697950 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:13.697965 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:13.699103 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:13.699106 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:13.700281 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:13.700358 (+ 77us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:13.700990 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:13.700995 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:13.701162 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:13.701780 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:13.701802 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:13.703557 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:13.705145 (+ 1588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:13.705152 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:13.705165 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:13.705401 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:13.705630 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:13.705632 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:13.705633 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:13.705760 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:13.706257 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:13.706262 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:13.706480 (+ 218us) client_negotiation.cc:769] Sending connection context
0808 06:05:13.706643 (+ 163us) client_negotiation.cc:240] Negotiation successful
0808 06:05:13.706885 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":131,"threads_started":1}
I20260808 06:05:13.709167 14933 heartbeater.cc:347] Connected to a master server at 127.1.151.190:34421
I20260808 06:05:13.709393 14933 heartbeater.cc:464] Registering TS with master...
I20260808 06:05:13.709873 14933 heartbeater.cc:511] Master 127.1.151.190:34421 requested a full tablet report, sending...
I20260808 06:05:13.711373 14629 ts_manager.cc:194] Registered new tserver with Master: 1e365f2563f941eb8eff0714bd52ee46 (127.1.151.129:40433)
I20260808 06:05:13.712625 14629 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.129@KRBTEST.COM'} at 127.1.151.129:43801
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab.
Entry for principal kudu/127.1.151.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab.
I20260808 06:05:13.723115 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.130:0
--local_ip_for_outbound_sockets=127.1.151.130
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:34421
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:39549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:05:13.832420 14942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:13.832659 14942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:13.832715 14942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:13.836594 14942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:13.836648 14942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:13.836723 14942 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.130
I20260808 06:05:13.841835 14942 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:39549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:34421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.130
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:05:13.842840 14942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:13.843847 14942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:13.853451 14952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:13.853611 14942 server_base.cc:1034] running on GCE node
W20260808 06:05:13.853451 14955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:13.853827 14953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:13.854236 14942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:13.854822 14942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:13.855981 14942 hybrid_clock.cc:648] HybridClock initialized: now 1786169113855961 us; error 30 us; skew 500 ppm
Aug 08 06:05:13 dist-test-slave-zj8q krb5kdc[14564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169113, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:13.858641 14942 init.cc:377] Logged in from keytab as kudu/127.1.151.130@KRBTEST.COM (short username kudu)
I20260808 06:05:13.859460 14942 webserver.cc:460] Webserver started at http://127.1.151.130:36481/ using document root <none> and password file <none>
I20260808 06:05:13.859973 14942 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:13.860038 14942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:13.860245 14942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:05:13.862222 14942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-1/data/instance:
uuid: "621e8aee844346379187a0cd241109ab"
format_stamp: "Formatted at 2026-08-08 06:05:13 on dist-test-slave-zj8q"
server_key: "4a71ddd80d928840e2fe3246db152bc2"
server_key_iv: "2bc917eb6a8a5fd17534195545c84441"
server_key_version: "encryptionkey@0"
I20260808 06:05:13.862754 14942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance:
uuid: "621e8aee844346379187a0cd241109ab"
format_stamp: "Formatted at 2026-08-08 06:05:13 on dist-test-slave-zj8q"
server_key: "4a71ddd80d928840e2fe3246db152bc2"
server_key_iv: "2bc917eb6a8a5fd17534195545c84441"
server_key_version: "encryptionkey@0"
I20260808 06:05:13.866324 14942 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260808 06:05:13.868690 14966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:13.869836 14942 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:05:13.869947 14942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-1/wal
uuid: "621e8aee844346379187a0cd241109ab"
format_stamp: "Formatted at 2026-08-08 06:05:13 on dist-test-slave-zj8q"
server_key: "4a71ddd80d928840e2fe3246db152bc2"
server_key_iv: "2bc917eb6a8a5fd17534195545c84441"
server_key_version: "encryptionkey@0"
I20260808 06:05:13.870039 14942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:13.897490 14942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:13.900686 14942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:05:13.900861 14942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:13.901491 14942 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:05:13.902367 14942 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:05:13.902419 14942 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:13.902475 14942 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:05:13.902501 14942 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:13.910825 14942 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.130:46225
I20260808 06:05:13.910878 15157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.130:46225 every 8 connection(s)
I20260808 06:05:13.911674 14942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
I20260808 06:05:13.919422 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 14942
I20260808 06:05:13.919548 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance
I20260808 06:05:13.919826 1630 external_mini_cluster.cc:1428] Setting key 605bf7f227b8a26ac8d4186cf13f01e8
Aug 08 06:05:13 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169113, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:05:13.927254 14689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:13.913396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:49513 (local address 127.1.151.190:34421)
0808 06:05:13.913537 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:13.913540 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:13.914545 (+ 1005us) server_negotiation.cc:407] Connection header received
0808 06:05:13.915392 (+ 847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:13.915394 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:13.915441 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:13.915510 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:13.916550 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:13.917062 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:13.918193 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:13.918414 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:13.918490 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:13.920773 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:13.920776 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:13.920778 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:13.920802 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:13.922417 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:13.925233 (+ 2816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:13.925237 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:13.925239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:13.925349 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:13.925846 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:13.925850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:13.925851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:13.926114 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:13.926217 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:13.926933 (+ 716us) server_negotiation.cc:299] Negotiation successful
0808 06:05:13.927079 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 06:05:13.929137 15162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:13.913742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.130:49513)
0808 06:05:13.914412 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:13.914452 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:13.915224 (+ 772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:13.915642 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:13.915648 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:13.915970 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:13.916437 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:13.916451 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:13.917170 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:13.917173 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:13.918069 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:13.918079 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:13.918531 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:13.918535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:13.918674 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:13.919165 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:13.919181 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:13.920635 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:13.922555 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:13.922561 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:13.922576 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:13.922940 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:13.925484 (+ 2544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:13.925488 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:13.925490 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:13.925711 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:13.926306 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:13.926313 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:13.926639 (+ 326us) client_negotiation.cc:769] Sending connection context
0808 06:05:13.927481 (+ 842us) client_negotiation.cc:240] Negotiation successful
0808 06:05:13.927851 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":111,"threads_started":1}
WARNING: no policy specified for kudu/127.1.151.131@KRBTEST.COM; defaulting to no policy
I20260808 06:05:13.933627 15158 heartbeater.cc:347] Connected to a master server at 127.1.151.190:34421
I20260808 06:05:13.933933 15158 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.131@KRBTEST.COM" created.
I20260808 06:05:13.934705 15158 heartbeater.cc:511] Master 127.1.151.190:34421 requested a full tablet report, sending...
I20260808 06:05:13.935936 14629 ts_manager.cc:194] Registered new tserver with Master: 621e8aee844346379187a0cd241109ab (127.1.151.130:46225)
I20260808 06:05:13.936532 14629 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.130@KRBTEST.COM'} at 127.1.151.130:49513
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab.
Entry for principal kudu/127.1.151.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab.
I20260808 06:05:13.950865 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.131:0
--local_ip_for_outbound_sockets=127.1.151.131
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:34421
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:39549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:05:14.063355 15171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:14.063592 15171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:14.063660 15171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:14.067613 15171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:14.067664 15171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:14.067736 15171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.131
I20260808 06:05:14.072778 15171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:39549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:34421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.131
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:05:14.073704 15171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:14.074728 15171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:14.083801 15182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:14.083848 15171 server_base.cc:1034] running on GCE node
W20260808 06:05:14.083758 15179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:14.083747 15180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:14.084403 15171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:14.084969 15171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:14.086133 15171 hybrid_clock.cc:648] HybridClock initialized: now 1786169114086113 us; error 33 us; skew 500 ppm
Aug 08 06:05:14 dist-test-slave-zj8q krb5kdc[14564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169114, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:14.088641 15171 init.cc:377] Logged in from keytab as kudu/127.1.151.131@KRBTEST.COM (short username kudu)
I20260808 06:05:14.089457 15171 webserver.cc:460] Webserver started at http://127.1.151.131:44847/ using document root <none> and password file <none>
I20260808 06:05:14.089974 15171 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:14.090039 15171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:14.090288 15171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:05:14.092267 15171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-2/data/instance:
uuid: "9c53166e6a864f9fa9b26fd897ba2bff"
format_stamp: "Formatted at 2026-08-08 06:05:14 on dist-test-slave-zj8q"
server_key: "481e82705f4554eb38dc23448ca17303"
server_key_iv: "963aa7f0f3a2b540959c52b33c959bf9"
server_key_version: "encryptionkey@0"
I20260808 06:05:14.092792 15171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance:
uuid: "9c53166e6a864f9fa9b26fd897ba2bff"
format_stamp: "Formatted at 2026-08-08 06:05:14 on dist-test-slave-zj8q"
server_key: "481e82705f4554eb38dc23448ca17303"
server_key_iv: "963aa7f0f3a2b540959c52b33c959bf9"
server_key_version: "encryptionkey@0"
I20260808 06:05:14.096312 15171 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260808 06:05:14.098623 15195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:14.099778 15171 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:05:14.099884 15171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-2/wal
uuid: "9c53166e6a864f9fa9b26fd897ba2bff"
format_stamp: "Formatted at 2026-08-08 06:05:14 on dist-test-slave-zj8q"
server_key: "481e82705f4554eb38dc23448ca17303"
server_key_iv: "963aa7f0f3a2b540959c52b33c959bf9"
server_key_version: "encryptionkey@0"
I20260808 06:05:14.099970 15171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:14.123380 15171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:14.126060 15171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:05:14.126258 15171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:14.126847 15171 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:05:14.127665 15171 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:05:14.127713 15171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:14.127767 15171 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:05:14.127796 15171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:14.136216 15171 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.131:39845
I20260808 06:05:14.136248 15378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.131:39845 every 8 connection(s)
I20260808 06:05:14.137068 15171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
Aug 08 06:05:14 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169114, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:05:14.147123 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 15171
I20260808 06:05:14.147253 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance
I20260808 06:05:14.147612 1630 external_mini_cluster.cc:1428] Setting key 6234a85a756f7ec112f6096ea68b5929
I20260808 06:05:14.150280 14689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:14.138872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:54875 (local address 127.1.151.190:34421)
0808 06:05:14.139010 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:14.139014 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:14.140018 (+ 1004us) server_negotiation.cc:407] Connection header received
0808 06:05:14.140912 (+ 894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:14.140916 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:14.140973 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:14.141046 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:14.142224 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:14.142762 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:14.143875 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:14.144098 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:14.144189 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:14.146589 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:14.146592 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:14.146594 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:14.146624 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:14.148154 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:14.148748 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:14.148752 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:14.148753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:14.148835 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:14.149179 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:14.149182 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:14.149184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:14.149367 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:14.149457 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:14.149949 (+ 492us) server_negotiation.cc:299] Negotiation successful
0808 06:05:14.150076 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 06:05:14.151087 15383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:14.139209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.131:54875)
0808 06:05:14.139875 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:14.139914 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:14.140714 (+ 800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:14.141239 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:14.141245 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:14.141585 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:14.142040 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:14.142054 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:14.142881 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:14.142884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:14.143755 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:14.143765 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:14.144235 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:14.144240 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:14.144398 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:14.144894 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:14.144910 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:14.146430 (+ 1520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:14.148276 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:14.148282 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:14.148295 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:14.148658 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:14.148931 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:14.148935 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:14.148937 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:14.149080 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:14.149470 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:14.149477 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:14.149722 (+ 245us) client_negotiation.cc:769] Sending connection context
0808 06:05:14.149885 (+ 163us) client_negotiation.cc:240] Negotiation successful
0808 06:05:14.150232 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":113,"threads_started":1}
I20260808 06:05:14.152196 15381 heartbeater.cc:347] Connected to a master server at 127.1.151.190:34421
I20260808 06:05:14.152424 15381 heartbeater.cc:464] Registering TS with master...
I20260808 06:05:14.152885 15381 heartbeater.cc:511] Master 127.1.151.190:34421 requested a full tablet report, sending...
I20260808 06:05:14.153983 14629 ts_manager.cc:194] Registered new tserver with Master: 9c53166e6a864f9fa9b26fd897ba2bff (127.1.151.131:39845)
I20260808 06:05:14.154768 14629 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.131@KRBTEST.COM'} at 127.1.151.131:54875
I20260808 06:05:14.161496 1630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.151.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.151.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.151.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.129.keytab.
Entry for principal oryx/127.1.151.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.129.keytab.
WARNING: no policy specified for oryx/127.1.151.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.151.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.151.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.130.keytab.
Entry for principal oryx/127.1.151.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.130.keytab.
WARNING: no policy specified for oryx/127.1.151.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.151.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.151.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.131.keytab.
Entry for principal oryx/127.1.151.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.131.keytab.
I20260808 06:05:14.233498 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 14722
I20260808 06:05:14.255594 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 14942
I20260808 06:05:14.264290 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 15171
I20260808 06:05:14.282917 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 14582
I20260808 06:05:14.298457 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:34421
--webserver_interface=127.1.151.190
--webserver_port=32905
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:39549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:34421
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:05:14.407874 15392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:14.408113 15392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:14.408155 15392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:14.412037 15392 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:05:14.412084 15392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:14.412110 15392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:14.412140 15392 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:05:14.412160 15392 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:05:14.417198 15392 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:39549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:34421
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:34421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=32905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:05:14.418120 15392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:14.419049 15392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:14.427004 15403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:14.427028 15406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:14.427081 15392 server_base.cc:1034] running on GCE node
W20260808 06:05:14.426999 15402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:14.427769 15392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:14.428789 15392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:14.429955 15392 hybrid_clock.cc:648] HybridClock initialized: now 1786169114429936 us; error 30 us; skew 500 ppm
Aug 08 06:05:14 dist-test-slave-zj8q krb5kdc[14564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169114, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:14.432664 15392 init.cc:377] Logged in from keytab as kudu/127.1.151.190@KRBTEST.COM (short username kudu)
I20260808 06:05:14.433461 15392 webserver.cc:460] Webserver started at http://127.1.151.190:32905/ using document root <none> and password file <none>
I20260808 06:05:14.433967 15392 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:14.434059 15392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:14.437743 15392 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:05:14.439772 15417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:14.440902 15392 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 06:05:14.441018 15392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "264f566d98374115870725477fb4c5da"
format_stamp: "Formatted at 2026-08-08 06:05:13 on dist-test-slave-zj8q"
server_key: "0229edc6789e2e860b8d52e35a75bd6c"
server_key_iv: "83015728d9275d62193d2aed008a2877"
server_key_version: "encryptionkey@0"
I20260808 06:05:14.441529 15392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:14.459542 15392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:14.462127 15392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:05:14.462314 15392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:14.468899 15392 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:34421
I20260808 06:05:14.468915 15500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:34421 every 8 connection(s)
I20260808 06:05:14.469923 15392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:05:14.474050 15502 sys_catalog.cc:263] Verifying existing consensus state
I20260808 06:05:14.474717 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 15392
I20260808 06:05:14.475121 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.129:40433
--local_ip_for_outbound_sockets=127.1.151.129
--tserver_master_addrs=127.1.151.190:34421
--webserver_port=44037
--webserver_interface=127.1.151.129
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:39549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.151.129
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 06:05:14.475729 15502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da: Bootstrap starting.
I20260808 06:05:14.492268 15502 log.cc:824] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da: Log is configured to *not* fsync() on all Append() calls
I20260808 06:05:14.499010 15502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 06:05:14.499358 15502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da: Bootstrap complete.
I20260808 06:05:14.503435 15502 raft_consensus.cc:348] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "264f566d98374115870725477fb4c5da" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34421 } }
I20260808 06:05:14.503830 15502 raft_consensus.cc:729] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 264f566d98374115870725477fb4c5da, State: Initialized, Role: FOLLOWER
I20260808 06:05:14.504302 15502 consensus_queue.cc:260] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "264f566d98374115870725477fb4c5da" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34421 } }
I20260808 06:05:14.504415 15502 raft_consensus.cc:388] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:05:14.504475 15502 raft_consensus.cc:482] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:05:14.504563 15502 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da [term 1 FOLLOWER]: Advancing to term 2
I20260808 06:05:14.506457 15502 raft_consensus.cc:504] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "264f566d98374115870725477fb4c5da" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34421 } }
I20260808 06:05:14.506765 15502 leader_election.cc:302] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 264f566d98374115870725477fb4c5da; no voters:
I20260808 06:05:14.507050 15502 leader_election.cc:288] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 06:05:14.507149 15509 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da [term 2 FOLLOWER]: Leader election won for term 2
I20260808 06:05:14.507344 15509 raft_consensus.cc:686] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da [term 2 LEADER]: Becoming Leader. State: Replica: 264f566d98374115870725477fb4c5da, State: Running, Role: LEADER
I20260808 06:05:14.507709 15509 consensus_queue.cc:237] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "264f566d98374115870725477fb4c5da" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34421 } }
I20260808 06:05:14.508275 15502 sys_catalog.cc:564] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:05:14.508635 15509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 264f566d98374115870725477fb4c5da. Latest consensus state: current_term: 2 leader_uuid: "264f566d98374115870725477fb4c5da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "264f566d98374115870725477fb4c5da" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34421 } } }
I20260808 06:05:14.508756 15509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da [sys.catalog]: This master's current role is: LEADER
I20260808 06:05:14.510049 15515 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:05:14.510088 15510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "264f566d98374115870725477fb4c5da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "264f566d98374115870725477fb4c5da" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34421 } } }
I20260808 06:05:14.510216 15510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da [sys.catalog]: This master's current role is: LEADER
I20260808 06:05:14.513087 15515 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:05:14.513736 15515 catalog_manager.cc:1259] Loaded cluster ID: f2fc48c20d2b4612bc2bae28c881b706
I20260808 06:05:14.513797 15515 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:05:14.516173 15515 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:05:14.516842 15515 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 264f566d98374115870725477fb4c5da: Loaded TSK: 0
I20260808 06:05:14.518025 15515 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 06:05:14.585554 15505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:14.585810 15505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:14.585870 15505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:14.589809 15505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:14.589869 15505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:14.589944 15505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.129
I20260808 06:05:14.595068 15505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:39549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.129.keytab
--principal=oryx/127.1.151.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.129:40433
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.151.129
--webserver_port=44037
--tserver_master_addrs=127.1.151.190:34421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.129
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:05:14.596057 15505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:14.597018 15505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:14.607359 15554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:14.607391 15551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:14.607530 15505 server_base.cc:1034] running on GCE node
W20260808 06:05:14.607625 15549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:14.608088 15505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:14.608700 15505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:14.609871 15505 hybrid_clock.cc:648] HybridClock initialized: now 1786169114609847 us; error 69 us; skew 500 ppm
Aug 08 06:05:14 dist-test-slave-zj8q krb5kdc[14564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169114, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:14.612776 15505 init.cc:377] Logged in from keytab as oryx/127.1.151.129@KRBTEST.COM (short username oryx)
I20260808 06:05:14.613628 15505 webserver.cc:460] Webserver started at http://127.1.151.129:44037/ using document root <none> and password file <none>
I20260808 06:05:14.614195 15505 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:14.614283 15505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:14.618127 15505 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 06:05:14.620232 15564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:14.621479 15505 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:05:14.621582 15505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-0/wal
uuid: "1e365f2563f941eb8eff0714bd52ee46"
format_stamp: "Formatted at 2026-08-08 06:05:13 on dist-test-slave-zj8q"
server_key: "8090adea8a05640808c2209f01707734"
server_key_iv: "77a36c5551b84e23934a0f3755cfa44a"
server_key_version: "encryptionkey@0"
I20260808 06:05:14.622089 15505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:14.654309 15505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:14.657178 15505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:05:14.657358 15505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:14.657935 15505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:05:14.658774 15505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:05:14.658823 15505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:14.658874 15505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:05:14.658905 15505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:14.667047 15505 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.129:40433
I20260808 06:05:14.667092 15773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.129:40433 every 8 connection(s)
I20260808 06:05:14.668033 15505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
I20260808 06:05:14.672432 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 15505
I20260808 06:05:14.672925 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.130:46225
--local_ip_for_outbound_sockets=127.1.151.130
--tserver_master_addrs=127.1.151.190:34421
--webserver_port=36481
--webserver_interface=127.1.151.130
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:39549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.151.130
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 06:05:14 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:14 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:14.681300 15778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:14.670068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.129:40641)
0808 06:05:14.670783 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:14.670833 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:14.671575 (+ 742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:14.672789 (+ 1214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:14.672797 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:14.673262 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:14.673863 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:14.673882 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:14.675578 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:14.675581 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:14.676427 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:14.676438 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:14.677449 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:14.677452 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:14.677580 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:14.678207 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:14.678223 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:14.680604 (+ 2381us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":114,"threads_started":1}
W20260808 06:05:14.681744 15775 heartbeater.cc:644] Failed to heartbeat to 127.1.151.190:34421 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.151.190:34421: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
W20260808 06:05:14.682322 15777 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:14.670018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:40641 (local address 127.1.151.190:34421)
0808 06:05:14.670502 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:14.670518 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:14.670932 (+ 414us) server_negotiation.cc:407] Connection header received
0808 06:05:14.671960 (+ 1028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:14.671972 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:14.672320 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:14.672711 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:14.674061 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:14.675428 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:14.676562 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:14.677300 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:14.677516 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:14.680915 (+ 3399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.129:40641: BlockingRecv error: recv got EOF from 127.1.151.129:40641 (error 108)
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":112,"threads_started":1}
Aug 08 06:05:14 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:14 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:14.687433 15778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:14.682211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.129:40777)
0808 06:05:14.682304 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:14.682317 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:14.682382 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:14.682886 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:14.682888 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:14.683013 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:14.683094 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:14.683099 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:14.684081 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:14.684083 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:14.684597 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:14.684604 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:14.685131 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:14.685133 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:14.685193 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:14.685659 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:14.685669 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:14.687350 (+ 1681us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 06:05:14.687810 15777 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:14.682302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:40777 (local address 127.1.151.190:34421)
0808 06:05:14.682665 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:14.682669 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:14.682685 (+ 16us) server_negotiation.cc:407] Connection header received
0808 06:05:14.682722 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:14.682724 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:14.682773 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:14.682860 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:14.683210 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:14.683974 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:14.684702 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:14.685025 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:14.685138 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:14.687502 (+ 2364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.129:40777: BlockingRecv error: recv got EOF from 127.1.151.129:40777 (error 108)
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":99,"threads_started":1}
Aug 08 06:05:14 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:14 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:14.692700 15778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:14.687874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.129:41269)
0808 06:05:14.688083 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:14.688096 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:14.688170 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:14.688366 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:14.688372 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:14.688483 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:14.688547 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:14.688551 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:14.689463 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:14.689465 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:14.689950 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:14.689956 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:14.690478 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:14.690481 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:14.690535 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:14.690926 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:14.690936 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:14.692618 (+ 1682us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147}
W20260808 06:05:14.692859 15775 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 06:05:14.693054 15777 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:14.687927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:41269 (local address 127.1.151.190:34421)
0808 06:05:14.688025 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:14.688029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:14.688148 (+ 119us) server_negotiation.cc:407] Connection header received
0808 06:05:14.688236 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:14.688239 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:14.688282 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:14.688359 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:14.688631 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:14.689379 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:14.690033 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:14.690388 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:14.690485 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:14.692761 (+ 2276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.129:41269: BlockingRecv error: recv got EOF from 127.1.151.129:41269 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260808 06:05:14.786860 15780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:14.787101 15780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:14.787139 15780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:14.790984 15780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:14.791035 15780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:14.791090 15780 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.130
I20260808 06:05:14.796141 15780 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:39549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.130.keytab
--principal=oryx/127.1.151.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.130:46225
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.151.130
--webserver_port=36481
--tserver_master_addrs=127.1.151.190:34421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.130
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:05:14.797078 15780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:14.798022 15780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:14.807359 15790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:14.807353 15800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:14.807353 15795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:14.807417 15780 server_base.cc:1034] running on GCE node
I20260808 06:05:14.807930 15780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:14.808531 15780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:14.809695 15780 hybrid_clock.cc:648] HybridClock initialized: now 1786169114809678 us; error 30 us; skew 500 ppm
Aug 08 06:05:14 dist-test-slave-zj8q krb5kdc[14564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169114, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:14.812398 15780 init.cc:377] Logged in from keytab as oryx/127.1.151.130@KRBTEST.COM (short username oryx)
I20260808 06:05:14.813215 15780 webserver.cc:460] Webserver started at http://127.1.151.130:36481/ using document root <none> and password file <none>
I20260808 06:05:14.813719 15780 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:14.813788 15780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:14.817485 15780 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260808 06:05:14.819664 15809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:14.820767 15780 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 06:05:14.820881 15780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-1/wal
uuid: "621e8aee844346379187a0cd241109ab"
format_stamp: "Formatted at 2026-08-08 06:05:13 on dist-test-slave-zj8q"
server_key: "4a71ddd80d928840e2fe3246db152bc2"
server_key_iv: "2bc917eb6a8a5fd17534195545c84441"
server_key_version: "encryptionkey@0"
I20260808 06:05:14.821396 15780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:14.849748 15780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:14.853009 15780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:05:14.853200 15780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:14.853804 15780 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:05:14.854655 15780 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:05:14.854703 15780 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:14.854758 15780 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:05:14.854786 15780 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:14.863101 15780 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.130:46225
I20260808 06:05:14.863129 16024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.130:46225 every 8 connection(s)
I20260808 06:05:14.864053 15780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
I20260808 06:05:14.870502 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 15780
I20260808 06:05:14.870909 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.131:39845
--local_ip_for_outbound_sockets=127.1.151.131
--tserver_master_addrs=127.1.151.190:34421
--webserver_port=44847
--webserver_interface=127.1.151.131
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:39549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.151.131
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 06:05:14 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:14 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 06:05:14.874421 15777 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:14.865784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:34367 (local address 127.1.151.190:34421)
0808 06:05:14.865949 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:14.865954 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:14.866977 (+ 1023us) server_negotiation.cc:407] Connection header received
0808 06:05:14.867778 (+ 801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:14.867781 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:14.867832 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:14.867903 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:14.868977 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:14.869498 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:14.870633 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:14.870853 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:14.870946 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:14.874318 (+ 3372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.130:34367: BlockingRecv error: recv got EOF from 127.1.151.130:34367 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 06:05:14.875066 16029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:14.866201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.130:34367)
0808 06:05:14.866840 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:14.866880 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:14.867615 (+ 735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:14.868053 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:14.868059 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:14.868400 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:14.868846 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:14.868859 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:14.869596 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:14.869599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:14.870505 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:14.870515 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:14.870992 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:14.870996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:14.871169 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:14.871682 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:14.871697 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:14.874185 (+ 2488us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":104,"threads_started":1}
W20260808 06:05:14.875504 16026 heartbeater.cc:644] Failed to heartbeat to 127.1.151.190:34421 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.151.190:34421: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Aug 08 06:05:14 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:14 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:14.880563 16029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:14.875888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.130:37593)
0808 06:05:14.876047 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:14.876060 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:14.876130 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:14.876342 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:14.876345 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:14.876482 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:14.876568 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:14.876573 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:14.877255 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:14.877259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:14.877884 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:14.877894 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:14.878355 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:14.878359 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:14.878453 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:14.878848 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:14.878859 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:14.880455 (+ 1596us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":97}
W20260808 06:05:14.880863 15777 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:14.876007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:37593 (local address 127.1.151.190:34421)
0808 06:05:14.876115 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:14.876119 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:14.876132 (+ 13us) server_negotiation.cc:407] Connection header received
0808 06:05:14.876215 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:14.876218 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:14.876257 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:14.876315 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:14.876676 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:14.877157 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:14.877996 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:14.878258 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:14.878349 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:14.880586 (+ 2237us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.130:37593: BlockingRecv error: recv got EOF from 127.1.151.130:37593 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 08 06:05:14 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:14 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:14.885732 16029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:14.880999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.130:57341)
0808 06:05:14.881126 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:14.881139 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:14.881207 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:14.881535 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:14.881537 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:14.881670 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:14.881733 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:14.881737 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:14.882695 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:14.882697 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:14.883184 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:14.883191 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:14.883705 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:14.883708 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:14.883759 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:14.884115 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:14.884126 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:14.885624 (+ 1498us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":80}
W20260808 06:05:14.885818 15777 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:14.881212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:57341 (local address 127.1.151.190:34421)
0808 06:05:14.881334 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:14.881339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:14.881354 (+ 15us) server_negotiation.cc:407] Connection header received
0808 06:05:14.881390 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:14.881393 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:14.881434 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:14.881525 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:14.881835 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:14.882586 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:14.883281 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:14.883615 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:14.883705 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:14.885732 (+ 2027us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.130:57341: BlockingRecv error: recv got EOF from 127.1.151.130:57341 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260808 06:05:14.886001 16026 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 06:05:14.983428 16030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:14.983676 16030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:14.983733 16030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:14.987653 16030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:14.987705 16030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:14.987782 16030 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.131
I20260808 06:05:14.992784 16030 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:39549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/krb5kdc/oryx_127.1.151.131.keytab
--principal=oryx/127.1.151.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.131:39845
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.151.131
--webserver_port=44847
--tserver_master_addrs=127.1.151.190:34421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.131
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:05:14.993721 16030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:14.994699 16030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:15.003765 16043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:15.003768 16040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:15.003983 16030 server_base.cc:1034] running on GCE node
W20260808 06:05:15.003829 16041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:15.004608 16030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:15.005174 16030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:15.006312 16030 hybrid_clock.cc:648] HybridClock initialized: now 1786169115006290 us; error 40 us; skew 500 ppm
Aug 08 06:05:15 dist-test-slave-zj8q krb5kdc[14564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169115, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.151.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:15.009038 16030 init.cc:377] Logged in from keytab as oryx/127.1.151.131@KRBTEST.COM (short username oryx)
I20260808 06:05:15.009990 16030 webserver.cc:460] Webserver started at http://127.1.151.131:44847/ using document root <none> and password file <none>
I20260808 06:05:15.010648 16030 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:15.010726 16030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:15.014494 16030 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 06:05:15.016412 16051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:15.017584 16030 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:05:15.017702 16030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-2/wal
uuid: "9c53166e6a864f9fa9b26fd897ba2bff"
format_stamp: "Formatted at 2026-08-08 06:05:14 on dist-test-slave-zj8q"
server_key: "481e82705f4554eb38dc23448ca17303"
server_key_iv: "963aa7f0f3a2b540959c52b33c959bf9"
server_key_version: "encryptionkey@0"
I20260808 06:05:15.018252 16030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:15.051079 16030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:15.054255 16030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:05:15.054431 16030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:15.055015 16030 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:05:15.055860 16030 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:05:15.055908 16030 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:15.055963 16030 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:05:15.055992 16030 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:15.064245 16030 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.131:39845
I20260808 06:05:15.064308 16225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.131:39845 every 8 connection(s)
I20260808 06:05:15.065241 16030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
I20260808 06:05:15.068571 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 16030
Aug 08 06:05:15 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:15 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 06:05:15.075866 15777 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:15.067059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:44127 (local address 127.1.151.190:34421)
0808 06:05:15.067189 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:15.067193 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:15.068176 (+ 983us) server_negotiation.cc:407] Connection header received
0808 06:05:15.069261 (+ 1085us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:15.069265 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:15.069324 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:15.069410 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:15.070708 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:15.071213 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:15.072316 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:15.072519 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:15.072601 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:15.075738 (+ 3137us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.131:44127: BlockingRecv error: recv got EOF from 127.1.151.131:44127 (error 108)
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":45}
I20260808 06:05:15.076434 16230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:15.067405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.131:44127)
0808 06:05:15.068037 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:15.068077 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:15.069027 (+ 950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:15.069711 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:15.069718 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:15.070050 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:15.070570 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:15.070584 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:15.071329 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:15.071332 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:15.072201 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:15.072210 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:15.072650 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:15.072654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:15.072807 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:15.073264 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:15.073279 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:15.075651 (+ 2372us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":122,"threads_started":1}
W20260808 06:05:15.076870 16226 heartbeater.cc:644] Failed to heartbeat to 127.1.151.190:34421 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.151.190:34421: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
I20260808 06:05:15.077759 15781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:15.069591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55922 (local address 127.1.151.190:34421)
0808 06:05:15.069723 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:15.069737 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:15.069749 (+ 12us) server_negotiation.cc:407] Connection header received
0808 06:05:15.069782 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:15.069784 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:15.069819 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:15.069890 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:15.070676 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:15.071215 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:15.072056 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:15.072275 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:15.072360 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:15.073764 (+ 1404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:15.073771 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:15.073788 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:15.073820 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:15.075488 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:15.075896 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:15.075902 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:15.075906 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:15.076065 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:15.076343 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:15.076347 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:15.076349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:15.076762 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:15.076954 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:15.077237 (+ 283us) server_negotiation.cc:299] Negotiation successful
0808 06:05:15.077577 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 06:05:15 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:15 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:15.081790 16230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:15.077354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.131:32787)
0808 06:05:15.077500 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:15.077510 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:15.077562 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:15.077775 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:15.077778 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:15.077916 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:15.078001 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:15.078006 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:15.078721 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:15.078725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:15.079233 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:15.079240 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:15.079695 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:15.079698 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:15.079782 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:15.080188 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:15.080198 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:15.081682 (+ 1484us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":89}
W20260808 06:05:15.081897 15777 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:15.077418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:32787 (local address 127.1.151.190:34421)
0808 06:05:15.077519 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:15.077522 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:15.077538 (+ 16us) server_negotiation.cc:407] Connection header received
0808 06:05:15.077621 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:15.077624 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:15.077664 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:15.077738 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:15.078108 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:15.078624 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:15.079338 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:15.079582 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:15.079671 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:15.081806 (+ 2135us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.131:32787: BlockingRecv error: recv got EOF from 127.1.151.131:32787 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 08 06:05:15 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:15 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:15.086510 16230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:15.082284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.131:51461)
0808 06:05:15.082367 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:15.082378 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:15.082449 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:15.082685 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:15.082689 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:15.082805 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:15.082866 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:15.082870 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:15.083522 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:15.083525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:15.084013 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:15.084022 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:15.084427 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:15.084431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:15.084503 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:15.084869 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:15.084879 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:15.086402 (+ 1523us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 06:05:15.086606 15777 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:15.082344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:51461 (local address 127.1.151.190:34421)
0808 06:05:15.082484 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:15.082488 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:15.082499 (+ 11us) server_negotiation.cc:407] Connection header received
0808 06:05:15.082529 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:15.082532 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:15.082564 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:15.082622 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:15.082964 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:15.083428 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:15.084126 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:15.084335 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:15.084416 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:15.086517 (+ 2101us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.131:51461: BlockingRecv error: recv got EOF from 127.1.151.131:51461 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260808 06:05:15.086748 16226 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 06:05:15 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:15 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:15.699496 16233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:15.693548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.129:34103)
0808 06:05:15.693803 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:15.693817 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:15.693919 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:15.694179 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:15.694182 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:15.694373 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:15.694500 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:15.694506 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:15.695620 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:15.695624 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:15.696384 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:15.696395 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:15.696821 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:15.696823 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:15.696889 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:15.697389 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:15.697404 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:15.699342 (+ 1938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":80,"threads_started":1}
W20260808 06:05:15.699618 16234 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:15.693669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:34103 (local address 127.1.151.190:34421)
0808 06:05:15.693929 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:15.693933 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:15.693970 (+ 37us) server_negotiation.cc:407] Connection header received
0808 06:05:15.694011 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:15.694013 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:15.694056 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:15.694125 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:15.694719 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:15.695457 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:15.696500 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:15.696734 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:15.696814 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:15.699483 (+ 2669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.129:34103: BlockingRecv error: recv got EOF from 127.1.151.129:34103 (error 108)
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":87,"threads_started":1}
Aug 08 06:05:15 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:15 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:15.891557 16235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:15.886564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.130:58207)
0808 06:05:15.886831 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:15.886844 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:15.886937 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:15.887130 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:15.887132 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:15.887313 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:15.887410 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:15.887416 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:15.888108 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:15.888111 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:15.888683 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:15.888691 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:15.889065 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:15.889067 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:15.889130 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:15.889597 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:15.889608 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:15.891437 (+ 1829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":92,"threads_started":1}
W20260808 06:05:15.891685 16234 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:15.886662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:58207 (local address 127.1.151.190:34421)
0808 06:05:15.886791 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:15.886794 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:15.886893 (+ 99us) server_negotiation.cc:407] Connection header received
0808 06:05:15.886999 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:15.887001 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:15.887040 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:15.887105 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:15.887492 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:15.888027 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:15.888771 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:15.888986 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:15.889056 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:15.891575 (+ 2519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.130:58207: BlockingRecv error: recv got EOF from 127.1.151.130:58207 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 08 06:05:16 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:16 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:16.092573 16238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:16.087289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.131:35463)
0808 06:05:16.087576 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:16.087592 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:16.087697 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:16.087943 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:16.087945 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:16.088159 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:16.088259 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:16.088265 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:16.089047 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:16.089051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:16.089628 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:16.089636 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:16.090085 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:16.090088 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:16.090192 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:16.090735 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:16.090747 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:16.092417 (+ 1670us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":84,"threads_started":1}
W20260808 06:05:16.092669 16234 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:16.087362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:35463 (local address 127.1.151.190:34421)
0808 06:05:16.087536 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:16.087550 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:16.087656 (+ 106us) server_negotiation.cc:407] Connection header received
0808 06:05:16.087792 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:16.087795 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:16.087845 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:16.087906 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:16.088365 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:16.088942 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:16.089740 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:16.089978 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:16.090066 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:16.092554 (+ 2488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.131:35463: BlockingRecv error: recv got EOF from 127.1.151.131:35463 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 08 06:05:16 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:16 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:16.705492 16240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:16.700406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.129:56105)
0808 06:05:16.700681 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:16.700694 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:16.700781 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:16.700985 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:16.700986 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:16.701171 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:16.701271 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:16.701276 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:16.702019 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:16.702022 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:16.702605 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:16.702615 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:16.703030 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:16.703033 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:16.703089 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:16.703523 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:16.703534 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:16.705342 (+ 1808us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":84,"threads_started":1}
W20260808 06:05:16.705710 16241 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:16.700471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:56105 (local address 127.1.151.190:34421)
0808 06:05:16.700793 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:16.700797 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:16.700810 (+ 13us) server_negotiation.cc:407] Connection header received
0808 06:05:16.700845 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:16.700847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:16.700887 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:16.700961 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:16.701372 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:16.701914 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:16.702718 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:16.702946 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:16.703025 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:16.705479 (+ 2454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.129:56105: BlockingRecv error: recv got EOF from 127.1.151.129:56105 (error 108)
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":164,"threads_started":1}
Aug 08 06:05:16 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:16 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:16.897815 16242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:16.892548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.130:45529)
0808 06:05:16.892852 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:16.892864 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:16.892973 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:16.893220 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:16.893224 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:16.893458 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:16.893557 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:16.893564 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:16.894373 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:16.894377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:16.894933 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:16.894942 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:16.895370 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:16.895374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:16.895457 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:16.895941 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:16.895953 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:16.897639 (+ 1686us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":109,"threads_started":1}
W20260808 06:05:16.897888 16241 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:16.892624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:45529 (local address 127.1.151.190:34421)
0808 06:05:16.892787 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:16.892792 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:16.892930 (+ 138us) server_negotiation.cc:407] Connection header received
0808 06:05:16.893060 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:16.893063 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:16.893112 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:16.893183 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:16.893698 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:16.894276 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:16.895044 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:16.895271 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:16.895356 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:16.897781 (+ 2425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.130:45529: BlockingRecv error: recv got EOF from 127.1.151.130:45529 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 06:05:17 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:17 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:17.098713 16245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:17.093419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.131:54621)
0808 06:05:17.093714 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:17.093728 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:17.093829 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:17.094037 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:17.094039 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:17.094273 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:17.094375 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:17.094381 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:17.095123 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:17.095127 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:17.095691 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:17.095700 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:17.096068 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:17.096071 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:17.096153 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:17.096585 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:17.096595 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:17.098517 (+ 1922us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":86,"threads_started":1}
W20260808 06:05:17.098762 16241 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:17.093561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:54621 (local address 127.1.151.190:34421)
0808 06:05:17.093691 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:17.093695 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:17.093777 (+ 82us) server_negotiation.cc:407] Connection header received
0808 06:05:17.093895 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:17.093897 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:17.093940 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:17.094007 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:17.094469 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:17.094985 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:17.095781 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:17.095985 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:17.096051 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:17.098663 (+ 2612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.131:54621: BlockingRecv error: recv got EOF from 127.1.151.131:54621 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 08 06:05:17 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:17 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:17.711407 16246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:17.706291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.129:34089)
0808 06:05:17.706561 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:17.706574 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:17.706680 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:17.706893 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:17.706895 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:17.707077 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:17.707187 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:17.707193 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:17.707880 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:17.707882 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:17.708466 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:17.708475 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:17.708981 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:17.708984 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:17.709053 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:17.709521 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:17.709531 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:17.711275 (+ 1744us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":89,"threads_started":1}
W20260808 06:05:17.711516 16249 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:17.706322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:34089 (local address 127.1.151.190:34421)
0808 06:05:17.706567 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:17.706571 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:17.706638 (+ 67us) server_negotiation.cc:407] Connection header received
0808 06:05:17.706749 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:17.706751 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:17.706795 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:17.706866 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:17.707272 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:17.707800 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:17.708548 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:17.708879 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:17.708957 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:17.711405 (+ 2448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.129:34089: BlockingRecv error: recv got EOF from 127.1.151.129:34089 (error 108)
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":75,"threads_started":1}
Aug 08 06:05:17 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:17 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:17.903852 16250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:17.898577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.130:51269)
0808 06:05:17.898852 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:17.898865 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:17.898961 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:17.899157 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:17.899159 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:17.899364 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:17.899464 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:17.899470 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:17.900189 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:17.900191 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:17.900755 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:17.900763 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:17.901251 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:17.901254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:17.901322 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:17.901835 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:17.901847 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:17.903733 (+ 1886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":103,"threads_started":1}
W20260808 06:05:17.903975 16249 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:17.898649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:51269 (local address 127.1.151.190:34421)
0808 06:05:17.898809 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:17.898814 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:17.898925 (+ 111us) server_negotiation.cc:407] Connection header received
0808 06:05:17.899030 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:17.899032 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:17.899076 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:17.899146 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:17.899559 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:17.900106 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:17.900859 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:17.901157 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:17.901237 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:17.903857 (+ 2620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.130:51269: BlockingRecv error: recv got EOF from 127.1.151.130:51269 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 06:05:18 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:18 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:18.105108 16251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:18.099544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.131:52277)
0808 06:05:18.099809 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:18.099822 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:18.099924 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:18.100260 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:18.100263 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:18.100450 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:18.100612 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:18.100619 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:18.101416 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:18.101420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:18.102018 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:18.102029 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:18.102512 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:18.102514 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:18.102578 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:18.103058 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:18.103069 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:18.104986 (+ 1917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":78,"threads_started":1}
W20260808 06:05:18.105275 16249 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:18.099639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:52277 (local address 127.1.151.190:34421)
0808 06:05:18.099775 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:18.099778 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:18.099925 (+ 147us) server_negotiation.cc:407] Connection header received
0808 06:05:18.100066 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:18.100069 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:18.100122 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:18.100193 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:18.100745 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:18.101281 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:18.102176 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:18.102420 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:18.102505 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:18.105149 (+ 2644us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.131:52277: BlockingRecv error: recv got EOF from 127.1.151.131:52277 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 06:05:18 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:18 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:18.721233 16252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:18.712102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.129:37651)
0808 06:05:18.712375 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:18.712389 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:18.712477 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:18.712703 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:18.712705 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:18.712886 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:18.712984 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:18.712989 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:18.713708 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:18.713710 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:18.714289 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:18.714297 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:18.714694 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:18.714696 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:18.714750 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:18.719100 (+ 4350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:18.719115 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:18.721110 (+ 1995us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":86,"threads_started":1}
W20260808 06:05:18.721350 16256 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:18.712189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:37651 (local address 127.1.151.190:34421)
0808 06:05:18.712433 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:18.712437 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:18.712449 (+ 12us) server_negotiation.cc:407] Connection header received
0808 06:05:18.712554 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:18.712556 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:18.712598 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:18.712679 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:18.713066 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:18.713591 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:18.714370 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:18.714590 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:18.714691 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:18.721234 (+ 6543us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.129:37651: BlockingRecv error: recv got EOF from 127.1.151.129:37651 (error 108)
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":80,"threads_started":1}
Aug 08 06:05:18 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:18 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:18.909823 16258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:18.904547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.130:48571)
0808 06:05:18.904846 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:18.904859 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:18.904952 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:18.905221 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:18.905224 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:18.905414 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:18.905517 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:18.905541 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:18.906304 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:18.906308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:18.906878 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:18.906887 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:18.907293 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:18.907296 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:18.907396 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:18.907851 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:18.907862 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:18.909677 (+ 1815us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":140,"threads_started":1}
W20260808 06:05:18.909931 16256 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:18.904628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:48571 (local address 127.1.151.190:34421)
0808 06:05:18.904764 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:18.904767 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:18.904914 (+ 147us) server_negotiation.cc:407] Connection header received
0808 06:05:18.905028 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:18.905031 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:18.905074 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:18.905129 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:18.905634 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:18.906196 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:18.906975 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:18.907182 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:18.907250 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:18.909814 (+ 2564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.130:48571: BlockingRecv error: recv got EOF from 127.1.151.130:48571 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 06:05:19 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:19 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:19.111334 16259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:19.105873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.131:44021)
0808 06:05:19.106189 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:19.106206 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:19.106316 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:19.106556 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:19.106559 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:19.106773 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:19.106872 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:19.106878 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:19.107619 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:19.107623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:19.108186 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:19.108194 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:19.108617 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:19.108620 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:19.108800 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:19.109290 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:19.109302 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:19.111195 (+ 1893us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":94,"threads_started":1}
W20260808 06:05:19.111431 16256 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:19.105960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:44021 (local address 127.1.151.190:34421)
0808 06:05:19.106110 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:19.106115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:19.106273 (+ 158us) server_negotiation.cc:407] Connection header received
0808 06:05:19.106404 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:19.106407 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:19.106455 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:19.106519 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:19.106984 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:19.107513 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:19.108298 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:19.108516 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:19.108603 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:19.111333 (+ 2730us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.131:44021: BlockingRecv error: recv got EOF from 127.1.151.131:44021 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 06:05:19 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:19 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:19.727658 16260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:19.722035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.129:33693)
0808 06:05:19.722310 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:19.722335 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:19.722445 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:19.722699 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:19.722701 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:19.722890 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:19.722984 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:19.722989 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:19.723741 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:19.723744 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:19.724323 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:19.724331 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:19.724818 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:19.724821 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:19.724889 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:19.725340 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:19.725354 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:19.727536 (+ 2182us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":109,"threads_started":1}
W20260808 06:05:19.727803 16261 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:19.722110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:33693 (local address 127.1.151.190:34421)
0808 06:05:19.722366 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:19.722369 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:19.722383 (+ 14us) server_negotiation.cc:407] Connection header received
0808 06:05:19.722535 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:19.722538 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:19.722584 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:19.722680 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:19.723071 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:19.723621 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:19.724405 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:19.724723 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:19.724799 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:19.727671 (+ 2872us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.129:33693: BlockingRecv error: recv got EOF from 127.1.151.129:33693 (error 108)
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":97,"threads_started":1}
Aug 08 06:05:19 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:19 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:19.915915 16262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:19.910631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.130:44063)
0808 06:05:19.910903 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:19.910919 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:19.911031 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:19.911316 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:19.911319 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:19.911529 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:19.911624 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:19.911630 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:19.912422 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:19.912426 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:19.912989 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:19.912998 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:19.913440 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:19.913444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:19.913534 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:19.914022 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:19.914035 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:19.915801 (+ 1766us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":83,"threads_started":1}
W20260808 06:05:19.916044 16261 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:19.910767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:44063 (local address 127.1.151.190:34421)
0808 06:05:19.910905 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:19.910909 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:19.910984 (+ 75us) server_negotiation.cc:407] Connection header received
0808 06:05:19.911120 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:19.911123 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:19.911201 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:19.911270 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:19.911769 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:19.912316 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:19.913103 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:19.913330 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:19.913417 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:19.915936 (+ 2519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.130:44063: BlockingRecv error: recv got EOF from 127.1.151.130:44063 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 06:05:20 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:20 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:20.117677 16263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:20.112129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.131:51509)
0808 06:05:20.112416 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:20.112430 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:20.112533 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:20.112763 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:20.112765 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:20.112956 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:20.113056 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:20.113061 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:20.113814 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:20.113817 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:20.114453 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:20.114464 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:20.114925 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:20.114929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:20.115016 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:20.115522 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:20.115536 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:20.117530 (+ 1994us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":99,"threads_started":1}
W20260808 06:05:20.117815 16261 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:20.112210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:51509 (local address 127.1.151.190:34421)
0808 06:05:20.112344 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:20.112348 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:20.112495 (+ 147us) server_negotiation.cc:407] Connection header received
0808 06:05:20.112613 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:20.112616 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:20.112665 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:20.112732 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:20.113176 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:20.113701 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:20.114591 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:20.114815 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:20.114908 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:20.117693 (+ 2785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.131:51509: BlockingRecv error: recv got EOF from 127.1.151.131:51509 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 06:05:20 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:20 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:20.733680 16264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:20.728308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.129:53887)
0808 06:05:20.728559 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:20.728573 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:20.728674 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:20.728898 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:20.728901 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:20.729090 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:20.729191 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:20.729196 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:20.729986 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:20.729990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:20.730581 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:20.730592 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:20.731047 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:20.731050 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:20.731132 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:20.731705 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:20.731717 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:20.733532 (+ 1815us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":72,"threads_started":1}
W20260808 06:05:20.733809 16266 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:20.728377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:53887 (local address 127.1.151.190:34421)
0808 06:05:20.728597 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:20.728601 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:20.728629 (+ 28us) server_negotiation.cc:407] Connection header received
0808 06:05:20.728747 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:20.728750 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:20.728792 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:20.728866 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:20.729277 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:20.729806 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:20.730696 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:20.730932 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:20.731035 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:20.733691 (+ 2656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.129:53887: BlockingRecv error: recv got EOF from 127.1.151.129:53887 (error 108)
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":66,"threads_started":1}
Aug 08 06:05:20 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:20 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:20.921576 16270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:20.916652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.130:37321)
0808 06:05:20.916939 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:20.916951 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:20.917049 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:20.917260 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:20.917262 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:20.917457 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:20.917552 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:20.917557 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:20.918276 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:20.918278 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:20.918844 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:20.918852 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:20.919264 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:20.919266 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:20.919323 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:20.919755 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:20.919766 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:20.921461 (+ 1695us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":94,"threads_started":1}
W20260808 06:05:20.921849 16266 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:20.916752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:37321 (local address 127.1.151.190:34421)
0808 06:05:20.916897 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:20.916903 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:20.917012 (+ 109us) server_negotiation.cc:407] Connection header received
0808 06:05:20.917118 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:20.917121 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:20.917167 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:20.917233 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:20.917633 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:20.918191 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:20.918946 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:20.919174 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:20.919254 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:20.921671 (+ 2417us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.130:37321: BlockingRecv error: recv got EOF from 127.1.151.130:37321 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 08 06:05:21 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:21 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:21.123674 16273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:21.118405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.131:60167)
0808 06:05:21.118700 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:21.118713 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:21.118816 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:21.119055 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:21.119057 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:21.119274 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:21.119378 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:21.119384 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:21.120089 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:21.120091 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:21.120684 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:21.120692 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:21.121209 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:21.121212 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:21.121277 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:21.121785 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:21.121797 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:21.123550 (+ 1753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":86,"threads_started":1}
W20260808 06:05:21.123806 16266 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:21.118497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:60167 (local address 127.1.151.190:34421)
0808 06:05:21.118629 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:21.118634 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:21.118772 (+ 138us) server_negotiation.cc:407] Connection header received
0808 06:05:21.118898 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:21.118901 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:21.118949 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:21.119005 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:21.119478 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:21.120008 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:21.120802 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:21.121055 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:21.121188 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:21.123687 (+ 2499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.131:60167: BlockingRecv error: recv got EOF from 127.1.151.131:60167 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 08 06:05:21 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:21 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:21.740069 16276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:21.734453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.129:60049)
0808 06:05:21.734766 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:21.734780 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:21.734883 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:21.735198 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:21.735200 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:21.735388 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:21.735487 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:21.735492 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:21.736244 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:21.736246 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:21.736804 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:21.736812 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:21.737268 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:21.737270 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:21.737327 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:21.737802 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:21.737814 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:21.739925 (+ 2111us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":71,"threads_started":1}
W20260808 06:05:21.740348 16277 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:21.734539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:60049 (local address 127.1.151.190:34421)
0808 06:05:21.734789 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:21.734793 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:21.734832 (+ 39us) server_negotiation.cc:407] Connection header received
0808 06:05:21.735008 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:21.735011 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:21.735066 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:21.735148 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:21.735593 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:21.736151 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:21.736933 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:21.737176 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:21.737267 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:21.740091 (+ 2824us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.129:60049: BlockingRecv error: recv got EOF from 127.1.151.129:60049 (error 108)
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":69,"threads_started":1}
Aug 08 06:05:21 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:21 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:21.927829 16278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:21.922458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.130:37887)
0808 06:05:21.922827 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:21.922841 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:21.922940 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:21.923207 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:21.923210 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:21.923457 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:21.923569 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:21.923577 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:21.924333 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:21.924337 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:21.924937 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:21.924945 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:21.925452 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:21.925455 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:21.925555 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:21.926031 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:21.926043 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:21.927714 (+ 1671us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":120,"threads_started":1}
W20260808 06:05:21.927959 16277 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:21.922530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:37887 (local address 127.1.151.190:34421)
0808 06:05:21.922727 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:21.922731 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:21.922898 (+ 167us) server_negotiation.cc:407] Connection header received
0808 06:05:21.923024 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:21.923027 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:21.923082 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:21.923166 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:21.923673 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:21.924211 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:21.925029 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:21.925315 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:21.925456 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:21.927845 (+ 2389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.130:37887: BlockingRecv error: recv got EOF from 127.1.151.130:37887 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 08 06:05:22 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:22 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:22.130246 16280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:22.124359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.131:52263)
0808 06:05:22.124687 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:22.124701 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:22.124867 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:22.125136 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:22.125139 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:22.125345 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:22.125456 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:22.125464 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:22.126376 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:22.126379 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:22.126935 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:22.126944 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:22.127401 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:22.127405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:22.127495 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:22.127969 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:22.127981 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:22.130079 (+ 2098us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":94,"threads_started":1}
W20260808 06:05:22.130358 16277 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:22.124429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:52263 (local address 127.1.151.190:34421)
0808 06:05:22.124561 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:22.124567 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:22.124855 (+ 288us) server_negotiation.cc:407] Connection header received
0808 06:05:22.124967 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:22.124970 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:22.125018 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:22.125082 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:22.125580 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:22.126273 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:22.127045 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:22.127273 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:22.127363 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:22.130255 (+ 2892us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.131:52263: BlockingRecv error: recv got EOF from 127.1.151.131:52263 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 08 06:05:22 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:22 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:22.746686 16281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:22.740914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.129:57159)
0808 06:05:22.741223 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:22.741237 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:22.741345 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:22.741581 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:22.741584 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:22.741772 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:22.741885 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:22.741891 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:22.742677 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:22.742681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:22.743434 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:22.743449 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:22.744020 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:22.744023 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:22.744108 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:22.744646 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:22.744657 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:22.746524 (+ 1867us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":86,"threads_started":1}
W20260808 06:05:22.746815 16283 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:22.741096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:57159 (local address 127.1.151.190:34421)
0808 06:05:22.741344 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:22.741347 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:22.741362 (+ 15us) server_negotiation.cc:407] Connection header received
0808 06:05:22.741417 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:22.741419 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:22.741461 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:22.741525 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:22.741984 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:22.742553 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:22.743563 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:22.743902 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:22.744003 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:22.746697 (+ 2694us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.129:57159: BlockingRecv error: recv got EOF from 127.1.151.129:57159 (error 108)
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":68,"threads_started":1}
Aug 08 06:05:22 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:22 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:22.933693 16284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:22.928531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.130:57625)
0808 06:05:22.928782 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:22.928795 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:22.928895 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:22.929102 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:22.929104 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:22.929304 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:22.929404 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:22.929410 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:22.930266 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:22.930269 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:22.930835 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:22.930844 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:22.931216 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:22.931219 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:22.931282 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:22.931738 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:22.931749 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:22.933546 (+ 1797us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":69,"threads_started":1}
W20260808 06:05:22.933807 16283 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:22.928597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:57625 (local address 127.1.151.190:34421)
0808 06:05:22.928723 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:22.928727 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:22.928856 (+ 129us) server_negotiation.cc:407] Connection header received
0808 06:05:22.928963 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:22.928965 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:22.929009 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:22.929080 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:22.929490 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:22.930142 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:22.930920 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:22.931135 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:22.931200 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:22.933690 (+ 2490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.130:57625: BlockingRecv error: recv got EOF from 127.1.151.130:57625 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 08 06:05:23 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:23 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:23.136222 16285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:23.130988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.131:49117)
0808 06:05:23.131252 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:23.131265 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:23.131377 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:23.131627 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:23.131630 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:23.131845 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:23.131943 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:23.131949 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:23.132696 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:23.132699 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:23.133271 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:23.133279 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:23.133717 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:23.133721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:23.133806 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:23.134301 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:23.134312 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:23.136038 (+ 1726us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":76,"threads_started":1}
W20260808 06:05:23.136294 16283 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:23.131092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:49117 (local address 127.1.151.190:34421)
0808 06:05:23.131212 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:23.131216 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:23.131345 (+ 129us) server_negotiation.cc:407] Connection header received
0808 06:05:23.131468 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:23.131471 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:23.131520 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:23.131586 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:23.132062 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:23.132591 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:23.133386 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:23.133608 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:23.133702 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:23.136190 (+ 2488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.131:49117: BlockingRecv error: recv got EOF from 127.1.151.131:49117 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 08 06:05:23 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:23 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:23.756407 16286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:23.747403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.129:49025)
0808 06:05:23.747664 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:23.747678 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:23.747779 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:23.747979 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:23.747981 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:23.748185 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:23.748284 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:23.748290 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:23.748982 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:23.748984 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:23.749548 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:23.749556 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:23.749945 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:23.749948 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:23.750006 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:23.754481 (+ 4475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:23.754492 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:23.756287 (+ 1795us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":78,"threads_started":1}
W20260808 06:05:23.756532 16290 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:23.747456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:49025 (local address 127.1.151.190:34421)
0808 06:05:23.747687 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:23.747691 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:23.747730 (+ 39us) server_negotiation.cc:407] Connection header received
0808 06:05:23.747840 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:23.747842 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:23.747883 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:23.747952 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:23.748371 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:23.748902 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:23.749640 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:23.749866 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:23.749937 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:23.756418 (+ 6481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.129:49025: BlockingRecv error: recv got EOF from 127.1.151.129:49025 (error 108)
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":69,"threads_started":1}
Aug 08 06:05:23 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:23 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:23.939538 16292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:23.934469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.130:52429)
0808 06:05:23.934767 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:23.934780 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:23.934881 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:23.935097 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:23.935098 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:23.935297 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:23.935394 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:23.935399 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:23.936082 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:23.936084 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:23.936652 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:23.936660 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:23.937046 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:23.937049 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:23.937132 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:23.937569 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:23.937580 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:23.939397 (+ 1817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":93,"threads_started":1}
W20260808 06:05:23.939663 16290 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:23.934559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:52429 (local address 127.1.151.190:34421)
0808 06:05:23.934713 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:23.934717 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:23.934845 (+ 128us) server_negotiation.cc:407] Connection header received
0808 06:05:23.934953 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:23.934955 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:23.935000 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:23.935068 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:23.935478 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:23.935999 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:23.936734 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:23.936951 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:23.937029 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:23.939540 (+ 2511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.130:52429: BlockingRecv error: recv got EOF from 127.1.151.130:52429 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 06:05:24 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:24 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:24.142252 16293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:24.137018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.131:55769)
0808 06:05:24.137297 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:24.137311 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:24.137409 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:24.137616 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:24.137619 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:24.137807 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:24.137903 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:24.137908 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:24.138630 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:24.138632 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:24.139183 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:24.139191 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:24.139679 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:24.139681 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:24.139741 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:24.140198 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:24.140208 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:24.142070 (+ 1862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":113,"threads_started":1}
W20260808 06:05:24.142338 16290 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:24.137079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:55769 (local address 127.1.151.190:34421)
0808 06:05:24.137218 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:24.137223 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:24.137375 (+ 152us) server_negotiation.cc:407] Connection header received
0808 06:05:24.137475 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:24.137477 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:24.137525 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:24.137588 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:24.137986 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:24.138550 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:24.139328 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:24.139575 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:24.139663 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:24.142236 (+ 2573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.131:55769: BlockingRecv error: recv got EOF from 127.1.151.131:55769 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 06:05:24 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:24 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:24.762989 16294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:24.757390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.129:40429)
0808 06:05:24.757779 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:24.757796 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:24.757909 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:24.758245 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:24.758248 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:24.758464 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:24.758575 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:24.758581 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:24.759363 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:24.759366 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:24.759990 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:24.760000 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:24.760445 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:24.760448 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:24.760569 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:24.761020 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:24.761035 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:24.762859 (+ 1824us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":82,"threads_started":1}
W20260808 06:05:24.763124 16295 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:24.757476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:40429 (local address 127.1.151.190:34421)
0808 06:05:24.757758 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:24.757763 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:24.757868 (+ 105us) server_negotiation.cc:407] Connection header received
0808 06:05:24.758023 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:24.758026 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:24.758084 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:24.758221 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:24.758676 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:24.759239 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:24.760100 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:24.760339 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:24.760414 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:24.762996 (+ 2582us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.129:40429: BlockingRecv error: recv got EOF from 127.1.151.129:40429 (error 108)
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":76,"threads_started":1}
Aug 08 06:05:24 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:24 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:24.946206 16296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:24.940411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.130:49743)
0808 06:05:24.940717 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:24.940732 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:24.940849 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:24.941125 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:24.941128 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:24.941364 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:24.941491 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:24.941500 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:24.942446 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:24.942449 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:24.943067 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:24.943080 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:24.943524 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:24.943526 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:24.943607 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:24.944085 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:24.944098 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:24.946040 (+ 1942us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":92,"threads_started":1}
W20260808 06:05:24.946318 16295 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:24.940517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:49743 (local address 127.1.151.190:34421)
0808 06:05:24.940681 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:24.940685 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:24.940809 (+ 124us) server_negotiation.cc:407] Connection header received
0808 06:05:24.940948 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:24.940950 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:24.940995 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:24.941061 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:24.941612 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:24.942320 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:24.943203 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:24.943430 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:24.943512 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:24.946210 (+ 2698us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.130:49743: BlockingRecv error: recv got EOF from 127.1.151.130:49743 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 06:05:25 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:25 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:25.148387 16297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:25.143036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.131:47447)
0808 06:05:25.143324 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:25.143339 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:25.143456 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:25.143698 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:25.143701 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:25.143912 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:25.144011 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:25.144017 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:25.144780 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:25.144784 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:25.145377 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:25.145386 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:25.145845 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:25.145849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:25.145935 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:25.146445 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:25.146457 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:25.148204 (+ 1747us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":85,"threads_started":1}
W20260808 06:05:25.148445 16295 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:25.143197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:47447 (local address 127.1.151.190:34421)
0808 06:05:25.143330 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:25.143335 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:25.143410 (+ 75us) server_negotiation.cc:407] Connection header received
0808 06:05:25.143544 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:25.143547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:25.143596 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:25.143662 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:25.144132 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:25.144667 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:25.145494 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:25.145733 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:25.145829 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:25.148344 (+ 2515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.131:47447: BlockingRecv error: recv got EOF from 127.1.151.131:47447 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 08 06:05:25 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:25 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:25.773276 16298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:25.763741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.129:57961)
0808 06:05:25.764016 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:25.764050 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:25.764173 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:25.764457 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:25.764460 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:25.764734 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:25.764856 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:25.764865 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:25.765794 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:25.765797 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:25.766392 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:25.766402 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:25.766778 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:25.766781 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:25.766836 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:25.771246 (+ 4410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:25.771260 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:25.773120 (+ 1860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":88,"threads_started":1}
W20260808 06:05:25.773392 16299 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:25.763796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:57961 (local address 127.1.151.190:34421)
0808 06:05:25.764031 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:25.764034 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:25.764127 (+ 93us) server_negotiation.cc:407] Connection header received
0808 06:05:25.764281 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:25.764284 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:25.764338 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:25.764424 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:25.764980 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:25.765692 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:25.766484 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:25.766704 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:25.766771 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:25.773272 (+ 6501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.129:57961: BlockingRecv error: recv got EOF from 127.1.151.129:57961 (error 108)
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":60,"threads_started":1}
Aug 08 06:05:25 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:25 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:25.955765 16300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:25.947056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.130:49219)
0808 06:05:25.947462 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:25.947477 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:25.947581 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:25.950764 (+ 3183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:25.950767 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:25.950975 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:25.951075 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:25.951082 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:25.951832 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:25.951836 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:25.952394 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:25.952403 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:25.953025 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:25.953028 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:25.953104 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:25.953622 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:25.953639 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:25.955615 (+ 1976us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":93,"threads_started":1}
W20260808 06:05:25.955940 16299 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:25.950424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:49219 (local address 127.1.151.190:34421)
0808 06:05:25.950550 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:25.950554 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:25.950565 (+ 11us) server_negotiation.cc:407] Connection header received
0808 06:05:25.950599 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:25.950601 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:25.950644 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:25.950715 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:25.951194 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:25.951725 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:25.952511 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:25.952887 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:25.953000 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:25.955814 (+ 2814us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.130:49219: BlockingRecv error: recv got EOF from 127.1.151.130:49219 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 08 06:05:26 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:26 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:26.154323 16301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:26.149207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.131:34977)
0808 06:05:26.149477 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:26.149491 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:26.149605 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:26.149830 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:26.149832 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:26.150018 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:26.150133 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:26.150138 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:26.150880 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:26.150883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:26.151452 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:26.151460 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:26.151859 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:26.151861 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:26.151924 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:26.152393 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:26.152404 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:26.154171 (+ 1767us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":79,"threads_started":1}
W20260808 06:05:26.154438 16299 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:26.149312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:34977 (local address 127.1.151.190:34421)
0808 06:05:26.149452 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:26.149455 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:26.149561 (+ 106us) server_negotiation.cc:407] Connection header received
0808 06:05:26.149689 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:26.149692 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:26.149738 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:26.149803 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:26.150251 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:26.150787 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:26.151539 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:26.151775 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:26.151848 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:26.154328 (+ 2480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.131:34977: BlockingRecv error: recv got EOF from 127.1.151.131:34977 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 06:05:26 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:26 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:26.779367 16304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:26.773975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.129:51505)
0808 06:05:26.774288 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:26.774304 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:26.774423 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:26.774675 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:26.774678 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:26.774880 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:26.774978 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:26.774984 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:26.775772 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:26.775776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:26.776335 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:26.776343 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:26.776782 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:26.776786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:26.776867 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:26.777332 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:26.777343 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:26.779226 (+ 1883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":90,"threads_started":1}
W20260808 06:05:26.779505 16305 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:26.774052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:51505 (local address 127.1.151.190:34421)
0808 06:05:26.774317 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:26.774322 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:26.774379 (+ 57us) server_negotiation.cc:407] Connection header received
0808 06:05:26.774511 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:26.774514 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:26.774563 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:26.774637 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:26.775093 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:26.775663 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:26.776446 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:26.776682 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:26.776770 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:26.779371 (+ 2601us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.129:51505: BlockingRecv error: recv got EOF from 127.1.151.129:51505 (error 108)
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":103,"threads_started":1}
Aug 08 06:05:26 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:26 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:26.962095 16308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:26.956758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.130:58687)
0808 06:05:26.957030 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:26.957043 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:26.957161 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:26.957426 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:26.957429 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:26.957612 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:26.957718 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:26.957724 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:26.958529 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:26.958532 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:26.959117 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:26.959127 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:26.959547 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:26.959550 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:26.959621 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:26.960128 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:26.960140 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:26.961933 (+ 1793us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":81,"threads_started":1}
W20260808 06:05:26.962193 16305 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:26.956829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:58687 (local address 127.1.151.190:34421)
0808 06:05:26.956955 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:26.956959 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:26.957100 (+ 141us) server_negotiation.cc:407] Connection header received
0808 06:05:26.957229 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:26.957232 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:26.957274 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:26.957341 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:26.957823 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:26.958424 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:26.959228 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:26.959457 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:26.959535 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:26.962068 (+ 2533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.130:58687: BlockingRecv error: recv got EOF from 127.1.151.130:58687 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 08 06:05:27 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:27 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:27.160467 16309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:27.155236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.131:35947)
0808 06:05:27.155496 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:27.155509 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:27.155609 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:27.155833 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:27.155835 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:27.156019 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:27.156138 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:27.156144 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:27.156848 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:27.156850 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:27.157426 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:27.157434 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:27.157880 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:27.157883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:27.157948 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:27.158447 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:27.158458 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:27.160349 (+ 1891us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":76,"threads_started":1}
W20260808 06:05:27.160601 16305 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:27.155335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:35947 (local address 127.1.151.190:34421)
0808 06:05:27.155472 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:27.155476 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:27.155575 (+ 99us) server_negotiation.cc:407] Connection header received
0808 06:05:27.155688 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:27.155690 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:27.155737 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:27.155802 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:27.156236 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:27.156763 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:27.157550 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:27.157786 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:27.157870 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:27.160482 (+ 2612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.131:35947: BlockingRecv error: recv got EOF from 127.1.151.131:35947 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 06:05:27 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:27 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:27.785192 16312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:27.780211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.129:53169)
0808 06:05:27.780589 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:27.780613 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:27.780710 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:27.780902 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:27.780904 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:27.781087 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:27.781187 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:27.781192 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:27.781893 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:27.781896 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:27.782483 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:27.782492 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:27.782853 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:27.782856 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:27.782913 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:27.783369 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:27.783379 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:27.785062 (+ 1683us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":83,"threads_started":1}
W20260808 06:05:27.785348 16313 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:27.780272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:53169 (local address 127.1.151.190:34421)
0808 06:05:27.780514 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:27.780517 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:27.780679 (+ 162us) server_negotiation.cc:407] Connection header received
0808 06:05:27.780773 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:27.780775 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:27.780819 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:27.780891 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:27.781271 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:27.781793 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:27.782568 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:27.782781 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:27.782846 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:27.785220 (+ 2374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.129:53169: BlockingRecv error: recv got EOF from 127.1.151.129:53169 (error 108)
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":61,"threads_started":1}
Aug 08 06:05:27 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:27 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:27.968102 16317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:27.962847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.130:45731)
0808 06:05:27.963132 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:27.963149 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:27.963271 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:27.963514 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:27.963517 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:27.963730 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:27.963823 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:27.963830 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:27.964573 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:27.964577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:27.965163 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:27.965172 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:27.965624 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:27.965628 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:27.965714 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:27.966179 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:27.966195 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:27.967986 (+ 1791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":106,"threads_started":1}
W20260808 06:05:27.968252 16313 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:27.962986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:45731 (local address 127.1.151.190:34421)
0808 06:05:27.963154 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:27.963158 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:27.963221 (+ 63us) server_negotiation.cc:407] Connection header received
0808 06:05:27.963356 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:27.963359 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:27.963407 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:27.963472 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:27.963936 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:27.964468 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:27.965280 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:27.965502 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:27.965602 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:27.968131 (+ 2529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.130:45731: BlockingRecv error: recv got EOF from 127.1.151.130:45731 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 06:05:28 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:28 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:28.166481 16318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:28.161252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.131:45849)
0808 06:05:28.161524 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:28.161538 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:28.161639 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:28.161843 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:28.161845 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:28.162029 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:28.162146 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:28.162181 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:28.162897 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:28.162901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:28.163494 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:28.163503 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:28.163897 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:28.163900 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:28.163959 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:28.164401 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:28.164412 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:28.166351 (+ 1939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":90,"threads_started":1}
W20260808 06:05:28.166556 16313 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:28.161330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:45849 (local address 127.1.151.190:34421)
0808 06:05:28.161481 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:28.161484 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:28.161590 (+ 106us) server_negotiation.cc:407] Connection header received
0808 06:05:28.161704 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:28.161706 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:28.161751 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:28.161816 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:28.162271 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:28.162803 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:28.163585 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:28.163815 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:28.163889 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:28.166463 (+ 2574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.131:45849: BlockingRecv error: recv got EOF from 127.1.151.131:45849 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 08 06:05:28 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:28 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:28.795255 16319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:28.785925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.129:40321)
0808 06:05:28.786268 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:28.786282 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:28.786387 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:28.786711 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:28.786715 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:28.786948 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:28.787074 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:28.787083 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:28.787972 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:28.787975 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:28.788535 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:28.788544 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:28.788944 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:28.788946 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:28.789004 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:28.793312 (+ 4308us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:28.793325 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:28.795082 (+ 1757us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":100,"threads_started":1}
W20260808 06:05:28.795351 16320 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:28.786007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:40321 (local address 127.1.151.190:34421)
0808 06:05:28.786331 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:28.786334 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:28.786347 (+ 13us) server_negotiation.cc:407] Connection header received
0808 06:05:28.786477 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:28.786480 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:28.786532 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:28.786689 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:28.787199 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:28.787870 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:28.788648 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:28.788865 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:28.788933 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:28.795235 (+ 6302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.129:40321: BlockingRecv error: recv got EOF from 127.1.151.129:40321 (error 108)
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":110,"threads_started":1}
Aug 08 06:05:28 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:28 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:28.974279 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:28.968825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.130:57811)
0808 06:05:28.969174 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:28.969190 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:28.969298 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:28.969577 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:28.969580 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:28.969807 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:28.969930 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:28.969939 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:28.970764 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:28.970766 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:28.971358 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:28.971366 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:28.971760 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:28.971763 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:28.971821 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:28.972280 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:28.972295 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:28.974090 (+ 1795us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":74,"threads_started":1}
W20260808 06:05:28.974349 16320 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:28.968935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:57811 (local address 127.1.151.190:34421)
0808 06:05:28.969088 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:28.969092 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:28.969255 (+ 163us) server_negotiation.cc:407] Connection header received
0808 06:05:28.969406 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:28.969409 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:28.969462 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:28.969522 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:28.970040 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:28.970665 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:28.971454 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:28.971681 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:28.971752 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:28.974249 (+ 2497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.130:57811: BlockingRecv error: recv got EOF from 127.1.151.130:57811 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 08 06:05:29 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:29 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.131@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:29.172647 16324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:29.167258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.131:55869)
0808 06:05:29.167541 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:29.167553 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:29.167668 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:29.167907 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:29.167909 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:29.168122 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:29.168221 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:29.168227 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:29.168961 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:29.168965 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:29.169540 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:29.169549 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:29.170014 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:29.170017 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:29.170104 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:29.170652 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:29.170664 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:29.172459 (+ 1795us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":93,"threads_started":1}
W20260808 06:05:29.172708 16320 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:29.167351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:55869 (local address 127.1.151.190:34421)
0808 06:05:29.167483 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:29.167488 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:29.167616 (+ 128us) server_negotiation.cc:407] Connection header received
0808 06:05:29.167754 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:29.167757 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:29.167803 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:29.167868 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:29.168334 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:29.168857 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:29.169682 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:29.169912 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:29.169999 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:29.172601 (+ 2602us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.131:55869: BlockingRecv error: recv got EOF from 127.1.151.131:55869 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 08 06:05:29 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:29 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.129@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:29.801568 16325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:29.796134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.129:52113)
0808 06:05:29.796395 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:29.796408 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:29.796491 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:29.796718 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:29.796720 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:29.796900 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:29.796995 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:29.797000 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:29.797823 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:29.797827 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:29.798597 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:29.798609 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:29.799082 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:29.799084 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:29.799151 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:29.799651 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:29.799668 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:29.801449 (+ 1781us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":83,"threads_started":1}
W20260808 06:05:29.801748 16326 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:29.796217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:52113 (local address 127.1.151.190:34421)
0808 06:05:29.796456 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:29.796459 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:29.796473 (+ 14us) server_negotiation.cc:407] Connection header received
0808 06:05:29.796574 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:29.796577 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:29.796619 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:29.796692 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:29.797092 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:29.797728 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:29.798719 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:29.798949 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:29.799032 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:29.801607 (+ 2575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.129:52113: BlockingRecv error: recv got EOF from 127.1.151.129:52113 (error 108)
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":60,"threads_started":1}
Aug 08 06:05:29 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 06:05:29 dist-test-slave-zj8q krb5kdc[14564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.151.130@KRBTEST.COM for oryx/127.1.151.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 06:05:29.980589 16327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:29.975201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34421 (local address 127.1.151.130:49639)
0808 06:05:29.975459 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:29.975473 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:29.975611 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:29.975832 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:29.975833 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:29.976026 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:29.976124 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:29.976129 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:29.976823 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:29.976827 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:29.977406 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:29.977415 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:29.977805 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:29.977808 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:29.977876 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:29.978408 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:29.978421 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:29.980439 (+ 2018us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.151.190:34421: Server oryx/127.1.151.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":72,"threads_started":1}
W20260808 06:05:29.980674 16326 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:29.975301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:49639 (local address 127.1.151.190:34421)
0808 06:05:29.975469 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:29.975473 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:29.975526 (+ 53us) server_negotiation.cc:407] Connection header received
0808 06:05:29.975690 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:29.975693 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:29.975735 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:29.975806 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:29.976207 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:29.976724 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:29.977491 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:29.977711 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:29.977780 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:29.980567 (+ 2787us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.151.130:49639: BlockingRecv error: recv got EOF from 127.1.151.130:49639 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 06:05:30.070291 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 15505
I20260808 06:05:30.082281 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 15780
I20260808 06:05:30.109091 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 16030
I20260808 06:05:30.120579 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 15392
2026-08-08T06:05:30Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18148 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T06:05:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:05:30Z Disabled control of system clock
I20260808 06:05:30.156663 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:44743
--webserver_interface=127.1.151.190
--webserver_port=0
--builtin_ntp_servers=127.1.151.148:39387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:44743
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 06:05:30.265357 16335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:30.265625 16335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:30.265664 16335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:30.269575 16335 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:05:30.269642 16335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:30.269657 16335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:30.269672 16335 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:05:30.269690 16335 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:05:30.274366 16335 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:39387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:44743
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:44743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:05:30.275368 16335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:30.276678 16335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 06:05:30.285252 16335 server_base.cc:1034] running on GCE node
W20260808 06:05:30.285194 16347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:30.285169 16350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:30.285175 16355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:30.285965 16335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:30.287027 16335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:30.288193 16335 hybrid_clock.cc:648] HybridClock initialized: now 1786169130288180 us; error 25 us; skew 500 ppm
I20260808 06:05:30.289942 16335 webserver.cc:460] Webserver started at http://127.1.151.190:44853/ using document root <none> and password file <none>
I20260808 06:05:30.290540 16335 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:30.290612 16335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:30.290823 16335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:05:30.292845 16335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "e5a80f208cc24133a3075e49378b0797"
format_stamp: "Formatted at 2026-08-08 06:05:30 on dist-test-slave-zj8q"
server_key: "582fee4d94b78d53dfffb68a38f7b174"
server_key_iv: "a93e485e85c6429249fb38d0cca3eaf8"
server_key_version: "encryptionkey@0"
I20260808 06:05:30.293376 16335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "e5a80f208cc24133a3075e49378b0797"
format_stamp: "Formatted at 2026-08-08 06:05:30 on dist-test-slave-zj8q"
server_key: "582fee4d94b78d53dfffb68a38f7b174"
server_key_iv: "a93e485e85c6429249fb38d0cca3eaf8"
server_key_version: "encryptionkey@0"
I20260808 06:05:30.297191 16335 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 06:05:30.299662 16362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:30.300886 16335 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 06:05:30.301012 16335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "e5a80f208cc24133a3075e49378b0797"
format_stamp: "Formatted at 2026-08-08 06:05:30 on dist-test-slave-zj8q"
server_key: "582fee4d94b78d53dfffb68a38f7b174"
server_key_iv: "a93e485e85c6429249fb38d0cca3eaf8"
server_key_version: "encryptionkey@0"
I20260808 06:05:30.301120 16335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:30.311484 16335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:30.312148 16335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:05:30.312304 16335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:30.318984 16335 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:44743
I20260808 06:05:30.318997 16446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:44743 every 8 connection(s)
I20260808 06:05:30.319861 16335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:05:30.322268 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 16335
I20260808 06:05:30.322381 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:05:30.322592 16447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:05:30.322655 1630 external_mini_cluster.cc:1428] Setting key 7205c467be9da779f5d59ca012dd9b5e
I20260808 06:05:30.329205 16447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5a80f208cc24133a3075e49378b0797: Bootstrap starting.
I20260808 06:05:30.331799 16447 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5a80f208cc24133a3075e49378b0797: Neither blocks nor log segments found. Creating new log.
I20260808 06:05:30.332358 16452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:30.324028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33512 (local address 127.1.151.190:44743)
0808 06:05:30.324640 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:30.324656 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:30.324690 (+ 34us) server_negotiation.cc:407] Connection header received
0808 06:05:30.325613 (+ 923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:30.325635 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:30.326016 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:30.326335 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:30.326658 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:30.327618 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:30.328816 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:30.329277 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:30.329423 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:30.330347 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:30.330354 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:30.330369 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:05:30.330848 (+ 479us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:30.330917 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:05:30.330942 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:30.331036 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:30.331184 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 06:05:30.331439 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":107,"threads_started":1}
I20260808 06:05:30.332664 16447 log.cc:824] T 00000000000000000000000000000000 P e5a80f208cc24133a3075e49378b0797: Log is configured to *not* fsync() on all Append() calls
I20260808 06:05:30.334636 16447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5a80f208cc24133a3075e49378b0797: No bootstrap required, opened a new log
I20260808 06:05:30.338012 16447 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5a80f208cc24133a3075e49378b0797 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5a80f208cc24133a3075e49378b0797" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44743 } }
I20260808 06:05:30.338282 16447 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5a80f208cc24133a3075e49378b0797 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:05:30.338342 16447 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5a80f208cc24133a3075e49378b0797 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5a80f208cc24133a3075e49378b0797, State: Initialized, Role: FOLLOWER
I20260808 06:05:30.338734 16447 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5a80f208cc24133a3075e49378b0797 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5a80f208cc24133a3075e49378b0797" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44743 } }
I20260808 06:05:30.338861 16447 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5a80f208cc24133a3075e49378b0797 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:05:30.338924 16447 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5a80f208cc24133a3075e49378b0797 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:05:30.339018 16447 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5a80f208cc24133a3075e49378b0797 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:05:30.339874 16447 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5a80f208cc24133a3075e49378b0797 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5a80f208cc24133a3075e49378b0797" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44743 } }
I20260808 06:05:30.340229 16447 leader_election.cc:302] T 00000000000000000000000000000000 P e5a80f208cc24133a3075e49378b0797 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5a80f208cc24133a3075e49378b0797; no voters:
I20260808 06:05:30.340554 16447 leader_election.cc:288] T 00000000000000000000000000000000 P e5a80f208cc24133a3075e49378b0797 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:05:30.340678 16457 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5a80f208cc24133a3075e49378b0797 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:05:30.340895 16457 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5a80f208cc24133a3075e49378b0797 [term 1 LEADER]: Becoming Leader. State: Replica: e5a80f208cc24133a3075e49378b0797, State: Running, Role: LEADER
I20260808 06:05:30.341228 16457 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5a80f208cc24133a3075e49378b0797 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5a80f208cc24133a3075e49378b0797" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44743 } }
I20260808 06:05:30.341699 16447 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5a80f208cc24133a3075e49378b0797 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:05:30.342904 16458 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5a80f208cc24133a3075e49378b0797 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5a80f208cc24133a3075e49378b0797" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5a80f208cc24133a3075e49378b0797" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44743 } } }
I20260808 06:05:30.343037 16458 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5a80f208cc24133a3075e49378b0797 [sys.catalog]: This master's current role is: LEADER
I20260808 06:05:30.343217 16460 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5a80f208cc24133a3075e49378b0797 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5a80f208cc24133a3075e49378b0797. Latest consensus state: current_term: 1 leader_uuid: "e5a80f208cc24133a3075e49378b0797" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5a80f208cc24133a3075e49378b0797" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44743 } } }
I20260808 06:05:30.343614 16471 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:05:30.343623 16460 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5a80f208cc24133a3075e49378b0797 [sys.catalog]: This master's current role is: LEADER
I20260808 06:05:30.346568 16471 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:05:30.352183 16471 catalog_manager.cc:1347] Generated new cluster ID: e002f1d708ef468fba182441aba54a97
I20260808 06:05:30.352254 16471 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:05:30.365276 16471 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:05:30.366523 16471 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:05:30.388419 16471 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5a80f208cc24133a3075e49378b0797: Generated new TSK 0
I20260808 06:05:30.389257 16471 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 06:05:30.393188 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.129:0
--local_ip_for_outbound_sockets=127.1.151.129
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:44743
--builtin_ntp_servers=127.1.151.148:39387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 06:05:30.503775 16487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:30.504072 16487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:30.504139 16487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:30.508360 16487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:30.508419 16487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:30.508491 16487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.129
I20260808 06:05:30.513150 16487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:39387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:44743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.129
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:05:30.514250 16487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:30.515656 16487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:30.525483 16495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:30.525483 16498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:30.525507 16494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:30.525848 16487 server_base.cc:1034] running on GCE node
I20260808 06:05:30.526257 16487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:30.526916 16487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:30.528088 16487 hybrid_clock.cc:648] HybridClock initialized: now 1786169130528064 us; error 36 us; skew 500 ppm
I20260808 06:05:30.529856 16487 webserver.cc:460] Webserver started at http://127.1.151.129:44895/ using document root <none> and password file <none>
I20260808 06:05:30.530463 16487 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:30.530536 16487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:30.530751 16487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:05:30.532755 16487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/data/instance:
uuid: "7b27585fa4c84b3688cfec6978c8aa46"
format_stamp: "Formatted at 2026-08-08 06:05:30 on dist-test-slave-zj8q"
server_key: "ed4142ba6bc3b0ba9634f3342e81165f"
server_key_iv: "b49d3ed7b327222963d4eff179937d14"
server_key_version: "encryptionkey@0"
I20260808 06:05:30.533294 16487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance:
uuid: "7b27585fa4c84b3688cfec6978c8aa46"
format_stamp: "Formatted at 2026-08-08 06:05:30 on dist-test-slave-zj8q"
server_key: "ed4142ba6bc3b0ba9634f3342e81165f"
server_key_iv: "b49d3ed7b327222963d4eff179937d14"
server_key_version: "encryptionkey@0"
I20260808 06:05:30.536878 16487 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260808 06:05:30.539389 16505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:30.540596 16487 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 06:05:30.540720 16487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/wal
uuid: "7b27585fa4c84b3688cfec6978c8aa46"
format_stamp: "Formatted at 2026-08-08 06:05:30 on dist-test-slave-zj8q"
server_key: "ed4142ba6bc3b0ba9634f3342e81165f"
server_key_iv: "b49d3ed7b327222963d4eff179937d14"
server_key_version: "encryptionkey@0"
I20260808 06:05:30.540818 16487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:30.562359 16487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:30.563030 16487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:05:30.563208 16487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:30.563802 16487 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:05:30.564743 16487 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:05:30.564792 16487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:30.564847 16487 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:05:30.564875 16487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:30.573249 16487 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.129:39137
I20260808 06:05:30.573277 16691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.129:39137 every 8 connection(s)
I20260808 06:05:30.574096 16487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
I20260808 06:05:30.580279 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 16487
I20260808 06:05:30.580390 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance
I20260808 06:05:30.580672 1630 external_mini_cluster.cc:1428] Setting key c76b689041e99a90bc1ed91e04ab3c75
I20260808 06:05:30.582751 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.130:0
--local_ip_for_outbound_sockets=127.1.151.130
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:44743
--builtin_ntp_servers=127.1.151.148:39387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 06:05:30.583341 16452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:30.575839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:49671 (local address 127.1.151.190:44743)
0808 06:05:30.575961 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:30.575965 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:30.576978 (+ 1013us) server_negotiation.cc:407] Connection header received
0808 06:05:30.578394 (+ 1416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:30.578398 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:30.578459 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:30.578531 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:30.579399 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:30.579976 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:30.581268 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:30.581537 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:30.581647 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:30.582819 (+ 1172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:30.582822 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:30.582824 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:05:30.582836 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:30.582861 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:05:30.582872 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:30.582933 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:30.583127 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 06:05:30.583198 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 06:05:30.584077 16696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:30.576184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:44743 (local address 127.1.151.129:49671)
0808 06:05:30.576829 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:30.576868 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:30.578177 (+ 1309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:30.578717 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:30.578723 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:30.578802 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:30.579259 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:30.579271 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:30.580107 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:30.580110 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:30.581105 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:30.581118 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:30.581706 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:30.581710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:30.581893 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:30.582494 (+ 601us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 06:05:30.582496 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:30.582551 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 06:05:30.582633 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:30.582975 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:30.582984 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:30.582989 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 06:05:30.583087 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 06:05:30.583308 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":124,"threads_started":1}
I20260808 06:05:30.585564 16693 heartbeater.cc:347] Connected to a master server at 127.1.151.190:44743
I20260808 06:05:30.585841 16693 heartbeater.cc:464] Registering TS with master...
I20260808 06:05:30.586364 16693 heartbeater.cc:511] Master 127.1.151.190:44743 requested a full tablet report, sending...
I20260808 06:05:30.588042 16381 ts_manager.cc:194] Registered new tserver with Master: 7b27585fa4c84b3688cfec6978c8aa46 (127.1.151.129:39137)
I20260808 06:05:30.589618 16381 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.151.129:49671
W20260808 06:05:30.692795 16697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:30.693048 16697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:30.693084 16697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:30.697005 16697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:30.697058 16697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:30.697108 16697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.130
I20260808 06:05:30.701695 16697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:39387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:44743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.130
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:05:30.702723 16697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:30.704031 16697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:30.713369 16704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:30.713376 16707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:30.713376 16705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:30.713586 16697 server_base.cc:1034] running on GCE node
I20260808 06:05:30.714035 16697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:30.714682 16697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:30.715847 16697 hybrid_clock.cc:648] HybridClock initialized: now 1786169130715824 us; error 37 us; skew 500 ppm
I20260808 06:05:30.717532 16697 webserver.cc:460] Webserver started at http://127.1.151.130:41119/ using document root <none> and password file <none>
I20260808 06:05:30.718091 16697 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:30.718186 16697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:30.718403 16697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:05:30.720441 16697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/data/instance:
uuid: "37a6588642f34884bd958f72674885c4"
format_stamp: "Formatted at 2026-08-08 06:05:30 on dist-test-slave-zj8q"
server_key: "5f8b1ac09a5f8c8232d1b35df5356077"
server_key_iv: "74931b392ec4f06fc7a7aa42f091cf35"
server_key_version: "encryptionkey@0"
I20260808 06:05:30.720974 16697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance:
uuid: "37a6588642f34884bd958f72674885c4"
format_stamp: "Formatted at 2026-08-08 06:05:30 on dist-test-slave-zj8q"
server_key: "5f8b1ac09a5f8c8232d1b35df5356077"
server_key_iv: "74931b392ec4f06fc7a7aa42f091cf35"
server_key_version: "encryptionkey@0"
I20260808 06:05:30.724604 16697 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260808 06:05:30.727072 16715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:30.728271 16697 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:05:30.728377 16697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/wal
uuid: "37a6588642f34884bd958f72674885c4"
format_stamp: "Formatted at 2026-08-08 06:05:30 on dist-test-slave-zj8q"
server_key: "5f8b1ac09a5f8c8232d1b35df5356077"
server_key_iv: "74931b392ec4f06fc7a7aa42f091cf35"
server_key_version: "encryptionkey@0"
I20260808 06:05:30.728471 16697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:30.746945 16697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:30.747589 16697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:05:30.747772 16697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:30.748333 16697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:05:30.749260 16697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:05:30.749308 16697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:30.749361 16697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:05:30.749387 16697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:30.757660 16697 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.130:38889
I20260808 06:05:30.757699 16892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.130:38889 every 8 connection(s)
I20260808 06:05:30.758538 16697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
I20260808 06:05:30.759858 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 16697
I20260808 06:05:30.759979 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance
I20260808 06:05:30.760346 1630 external_mini_cluster.cc:1428] Setting key 75a130eab075a6a818fb9977df1f4a5d
I20260808 06:05:30.762434 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.131:0
--local_ip_for_outbound_sockets=127.1.151.131
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:44743
--builtin_ntp_servers=127.1.151.148:39387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 06:05:30.767758 16452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:30.760309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:49941 (local address 127.1.151.190:44743)
0808 06:05:30.760439 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:30.760444 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:30.761749 (+ 1305us) server_negotiation.cc:407] Connection header received
0808 06:05:30.763222 (+ 1473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:30.763226 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:30.763281 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:30.763365 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:30.764185 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:30.764707 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:30.765823 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:30.766109 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:30.766251 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:30.767267 (+ 1016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:30.767271 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:30.767275 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:05:30.767290 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:30.767312 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:05:30.767322 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:30.767398 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:30.767529 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 06:05:30.767598 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 06:05:30.768514 16895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:30.760821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:44743 (local address 127.1.151.130:49941)
0808 06:05:30.761590 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:30.761646 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:30.763015 (+ 1369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:30.763506 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:30.763513 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:30.763590 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:30.764041 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:30.764053 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:30.764827 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:30.764830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:30.765705 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:30.765716 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:30.766296 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:30.766300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:30.766455 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:30.766996 (+ 541us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 06:05:30.766998 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:30.767036 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 06:05:30.767099 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:30.767396 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:30.767400 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:30.767404 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 06:05:30.767518 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 06:05:30.767741 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":146,"threads_started":1}
I20260808 06:05:30.769579 16893 heartbeater.cc:347] Connected to a master server at 127.1.151.190:44743
I20260808 06:05:30.769838 16893 heartbeater.cc:464] Registering TS with master...
I20260808 06:05:30.770463 16893 heartbeater.cc:511] Master 127.1.151.190:44743 requested a full tablet report, sending...
I20260808 06:05:30.771569 16381 ts_manager.cc:194] Registered new tserver with Master: 37a6588642f34884bd958f72674885c4 (127.1.151.130:38889)
I20260808 06:05:30.772109 16381 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.151.130:49941
W20260808 06:05:30.874135 16897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:30.874408 16897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:30.874451 16897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:30.878345 16897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:30.878387 16897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:30.878448 16897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.131
I20260808 06:05:30.882988 16897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:39387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:44743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.131
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:05:30.884023 16897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:30.885368 16897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:30.894672 16905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:30.894650 16902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:30.894801 16897 server_base.cc:1034] running on GCE node
W20260808 06:05:30.894948 16907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:30.895305 16897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:30.895874 16897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:30.897015 16897 hybrid_clock.cc:648] HybridClock initialized: now 1786169130896989 us; error 41 us; skew 500 ppm
I20260808 06:05:30.898684 16897 webserver.cc:460] Webserver started at http://127.1.151.131:40025/ using document root <none> and password file <none>
I20260808 06:05:30.899204 16897 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:30.899262 16897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:30.899451 16897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:05:30.901372 16897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/data/instance:
uuid: "f873281e4a8e458c9c8181345954f1de"
format_stamp: "Formatted at 2026-08-08 06:05:30 on dist-test-slave-zj8q"
server_key: "dfce956073c84377a33b88d8095291f8"
server_key_iv: "c3b6451b390d67aa0efce9071029d435"
server_key_version: "encryptionkey@0"
I20260808 06:05:30.901895 16897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance:
uuid: "f873281e4a8e458c9c8181345954f1de"
format_stamp: "Formatted at 2026-08-08 06:05:30 on dist-test-slave-zj8q"
server_key: "dfce956073c84377a33b88d8095291f8"
server_key_iv: "c3b6451b390d67aa0efce9071029d435"
server_key_version: "encryptionkey@0"
I20260808 06:05:30.905333 16897 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 06:05:30.907730 16919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:30.908852 16897 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 06:05:30.908953 16897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/wal
uuid: "f873281e4a8e458c9c8181345954f1de"
format_stamp: "Formatted at 2026-08-08 06:05:30 on dist-test-slave-zj8q"
server_key: "dfce956073c84377a33b88d8095291f8"
server_key_iv: "c3b6451b390d67aa0efce9071029d435"
server_key_version: "encryptionkey@0"
I20260808 06:05:30.909067 16897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:30.920624 16897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:30.921178 16897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:05:30.921316 16897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:30.921808 16897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:05:30.922710 16897 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:05:30.922758 16897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:30.922796 16897 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:05:30.922809 16897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:30.930907 16897 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.131:39771
I20260808 06:05:30.930960 17094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.131:39771 every 8 connection(s)
I20260808 06:05:30.931751 16897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
I20260808 06:05:30.939687 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 16897
I20260808 06:05:30.939800 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance
I20260808 06:05:30.940090 1630 external_mini_cluster.cc:1428] Setting key f5e4bf4a59e2695d8911a2f22378bbd2
I20260808 06:05:30.940208 16452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:30.933420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:45565 (local address 127.1.151.190:44743)
0808 06:05:30.933597 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:30.933603 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:30.934599 (+ 996us) server_negotiation.cc:407] Connection header received
0808 06:05:30.935870 (+ 1271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:30.935873 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:30.935920 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:30.935990 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:30.936796 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:30.937311 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:30.938387 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:30.938602 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:30.938679 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:30.939587 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:30.939591 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:30.939594 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:05:30.939609 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:30.939635 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:05:30.939660 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:30.939741 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:30.939989 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 06:05:30.940055 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 06:05:30.941128 17097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:30.933806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:44743 (local address 127.1.151.131:45565)
0808 06:05:30.934461 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:30.934500 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:30.935698 (+ 1198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:30.936152 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:30.936158 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:30.936239 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:30.936691 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:30.936703 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:30.937397 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:30.937400 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:30.938295 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:30.938305 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:30.938713 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:30.938715 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:30.938850 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:30.939333 (+ 483us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 06:05:30.939334 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:30.939369 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 06:05:30.939429 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:30.939822 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:30.939833 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:30.939840 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 06:05:30.939960 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 06:05:30.940206 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":114,"threads_started":1}
I20260808 06:05:30.942257 17095 heartbeater.cc:347] Connected to a master server at 127.1.151.190:44743
I20260808 06:05:30.942464 17095 heartbeater.cc:464] Registering TS with master...
I20260808 06:05:30.942960 17095 heartbeater.cc:511] Master 127.1.151.190:44743 requested a full tablet report, sending...
I20260808 06:05:30.943929 16381 ts_manager.cc:194] Registered new tserver with Master: f873281e4a8e458c9c8181345954f1de (127.1.151.131:39771)
I20260808 06:05:30.944540 16381 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.151.131:45565
I20260808 06:05:30.953860 1630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 06:05:30.956651 16452 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:30.955741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33522 (local address 127.1.151.190:44743)
0808 06:05:30.955863 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:30.955866 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:30.956006 (+ 140us) server_negotiation.cc:407] Connection header received
0808 06:05:30.956133 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:30.956136 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:30.956179 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:30.956237 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:30.956519 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33522: BlockingRecv error: recv got EOF from 127.0.0.1:33522 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 06:05:30.956775 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 16487
I20260808 06:05:30.972416 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 16697
I20260808 06:05:30.997542 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 16897
I20260808 06:05:31.010341 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 16335
2026-08-08T06:05:31Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (886 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T06:05:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:05:31Z Disabled control of system clock
I20260808 06:05:31.042999 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:45285
--webserver_interface=127.1.151.190
--webserver_port=0
--builtin_ntp_servers=127.1.151.148:33381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:45285
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 06:05:31.151402 17107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:31.151655 17107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:31.151690 17107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:31.155686 17107 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:05:31.155745 17107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:31.155763 17107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:31.155779 17107 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:05:31.155793 17107 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:05:31.160475 17107 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:33381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:45285
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:45285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17107
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:05:31.161499 17107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:31.162894 17107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:31.171020 17117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:31.171030 17121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:31.171062 17115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:31.171409 17107 server_base.cc:1034] running on GCE node
I20260808 06:05:31.172001 17107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:31.173054 17107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:31.174232 17107 hybrid_clock.cc:648] HybridClock initialized: now 1786169131174207 us; error 31 us; skew 500 ppm
I20260808 06:05:31.176124 17107 webserver.cc:460] Webserver started at http://127.1.151.190:45829/ using document root <none> and password file <none>
I20260808 06:05:31.176688 17107 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:31.176759 17107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:31.176956 17107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:05:31.179057 17107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "8c5dc4a441bd4a51841d13bf04542dfa"
format_stamp: "Formatted at 2026-08-08 06:05:31 on dist-test-slave-zj8q"
server_key: "90e6e7c04d5642f191062727eea4bf15"
server_key_iv: "74cc10f031f08dfb2568680515b16497"
server_key_version: "encryptionkey@0"
I20260808 06:05:31.179627 17107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "8c5dc4a441bd4a51841d13bf04542dfa"
format_stamp: "Formatted at 2026-08-08 06:05:31 on dist-test-slave-zj8q"
server_key: "90e6e7c04d5642f191062727eea4bf15"
server_key_iv: "74cc10f031f08dfb2568680515b16497"
server_key_version: "encryptionkey@0"
I20260808 06:05:31.183208 17107 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 06:05:31.185482 17128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:31.186663 17107 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:05:31.186776 17107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "8c5dc4a441bd4a51841d13bf04542dfa"
format_stamp: "Formatted at 2026-08-08 06:05:31 on dist-test-slave-zj8q"
server_key: "90e6e7c04d5642f191062727eea4bf15"
server_key_iv: "74cc10f031f08dfb2568680515b16497"
server_key_version: "encryptionkey@0"
I20260808 06:05:31.186866 17107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:31.191195 17107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:31.191747 17107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:05:31.191895 17107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:31.198448 17107 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:45285
I20260808 06:05:31.198459 17206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:45285 every 8 connection(s)
I20260808 06:05:31.199294 17107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:05:31.202338 17208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:05:31.207772 17208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c5dc4a441bd4a51841d13bf04542dfa: Bootstrap starting.
I20260808 06:05:31.208366 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 17107
I20260808 06:05:31.208474 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:05:31.208766 1630 external_mini_cluster.cc:1428] Setting key bacccdea677c68dbbb2c0d0dc48e953f
I20260808 06:05:31.210631 17208 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c5dc4a441bd4a51841d13bf04542dfa: Neither blocks nor log segments found. Creating new log.
I20260808 06:05:31.211508 17208 log.cc:824] T 00000000000000000000000000000000 P 8c5dc4a441bd4a51841d13bf04542dfa: Log is configured to *not* fsync() on all Append() calls
I20260808 06:05:31.213804 17208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c5dc4a441bd4a51841d13bf04542dfa: No bootstrap required, opened a new log
I20260808 06:05:31.214726 17211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:31.210111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36832 (local address 127.1.151.190:45285)
0808 06:05:31.210893 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:31.210914 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:31.210958 (+ 44us) server_negotiation.cc:407] Connection header received
0808 06:05:31.211787 (+ 829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:31.211814 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:31.212162 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:31.212495 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:31.213163 (+ 668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:31.213169 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:31.213182 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:05:31.213194 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:31.213260 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:05:31.213282 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:31.213394 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:31.213554 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 06:05:31.213808 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":122,"threads_started":1}
I20260808 06:05:31.217814 17208 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c5dc4a441bd4a51841d13bf04542dfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c5dc4a441bd4a51841d13bf04542dfa" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 45285 } }
I20260808 06:05:31.218029 17208 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c5dc4a441bd4a51841d13bf04542dfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:05:31.218062 17208 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c5dc4a441bd4a51841d13bf04542dfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c5dc4a441bd4a51841d13bf04542dfa, State: Initialized, Role: FOLLOWER
I20260808 06:05:31.218479 17208 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c5dc4a441bd4a51841d13bf04542dfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c5dc4a441bd4a51841d13bf04542dfa" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 45285 } }
I20260808 06:05:31.218602 17208 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c5dc4a441bd4a51841d13bf04542dfa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:05:31.218648 17208 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c5dc4a441bd4a51841d13bf04542dfa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:05:31.218716 17208 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c5dc4a441bd4a51841d13bf04542dfa [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:05:31.219671 17208 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c5dc4a441bd4a51841d13bf04542dfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c5dc4a441bd4a51841d13bf04542dfa" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 45285 } }
I20260808 06:05:31.219991 17208 leader_election.cc:302] T 00000000000000000000000000000000 P 8c5dc4a441bd4a51841d13bf04542dfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c5dc4a441bd4a51841d13bf04542dfa; no voters:
I20260808 06:05:31.220273 17208 leader_election.cc:288] T 00000000000000000000000000000000 P 8c5dc4a441bd4a51841d13bf04542dfa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:05:31.220422 17213 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c5dc4a441bd4a51841d13bf04542dfa [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:05:31.220639 17213 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c5dc4a441bd4a51841d13bf04542dfa [term 1 LEADER]: Becoming Leader. State: Replica: 8c5dc4a441bd4a51841d13bf04542dfa, State: Running, Role: LEADER
I20260808 06:05:31.220927 17213 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c5dc4a441bd4a51841d13bf04542dfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c5dc4a441bd4a51841d13bf04542dfa" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 45285 } }
I20260808 06:05:31.221402 17208 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c5dc4a441bd4a51841d13bf04542dfa [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:05:31.222622 17215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c5dc4a441bd4a51841d13bf04542dfa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c5dc4a441bd4a51841d13bf04542dfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c5dc4a441bd4a51841d13bf04542dfa" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 45285 } } }
I20260808 06:05:31.222633 17216 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c5dc4a441bd4a51841d13bf04542dfa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c5dc4a441bd4a51841d13bf04542dfa. Latest consensus state: current_term: 1 leader_uuid: "8c5dc4a441bd4a51841d13bf04542dfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c5dc4a441bd4a51841d13bf04542dfa" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 45285 } } }
I20260808 06:05:31.222818 17216 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c5dc4a441bd4a51841d13bf04542dfa [sys.catalog]: This master's current role is: LEADER
I20260808 06:05:31.223088 17215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c5dc4a441bd4a51841d13bf04542dfa [sys.catalog]: This master's current role is: LEADER
I20260808 06:05:31.223790 17230 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:05:31.226429 17230 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:05:31.232005 17230 catalog_manager.cc:1347] Generated new cluster ID: 6009fb1eb3664941934ab33b6b2f5e94
I20260808 06:05:31.232059 17230 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:05:31.244298 17230 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:05:31.244618 17230 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:05:31.262194 17230 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c5dc4a441bd4a51841d13bf04542dfa: Generated new TSK 0
I20260808 06:05:31.262882 17230 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 06:05:31.266482 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.129:0
--local_ip_for_outbound_sockets=127.1.151.129
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:45285
--builtin_ntp_servers=127.1.151.148:33381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 06:05:31.379686 17241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:31.379926 17241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:31.379963 17241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:31.383890 17241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:31.383939 17241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:31.384001 17241 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.129
I20260808 06:05:31.388645 17241 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:33381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:45285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.129
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:05:31.389647 17241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:31.391022 17241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:31.400277 17247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:31.400300 17248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:31.400288 17254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:31.400410 17241 server_base.cc:1034] running on GCE node
I20260808 06:05:31.400940 17241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:31.401546 17241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:31.402716 17241 hybrid_clock.cc:648] HybridClock initialized: now 1786169131402697 us; error 26 us; skew 500 ppm
I20260808 06:05:31.404479 17241 webserver.cc:460] Webserver started at http://127.1.151.129:34825/ using document root <none> and password file <none>
I20260808 06:05:31.405020 17241 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:31.405087 17241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:31.405290 17241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:05:31.407299 17241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/data/instance:
uuid: "dd2740fad39743bb98e346e2e99aef6e"
format_stamp: "Formatted at 2026-08-08 06:05:31 on dist-test-slave-zj8q"
server_key: "d1687d5a3e2c134de8654da3354eaf58"
server_key_iv: "25ea16b0bf05944ef3320a5403caa533"
server_key_version: "encryptionkey@0"
I20260808 06:05:31.407815 17241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance:
uuid: "dd2740fad39743bb98e346e2e99aef6e"
format_stamp: "Formatted at 2026-08-08 06:05:31 on dist-test-slave-zj8q"
server_key: "d1687d5a3e2c134de8654da3354eaf58"
server_key_iv: "25ea16b0bf05944ef3320a5403caa533"
server_key_version: "encryptionkey@0"
I20260808 06:05:31.411285 17241 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260808 06:05:31.413726 17260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:31.414892 17241 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:05:31.415007 17241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/wal
uuid: "dd2740fad39743bb98e346e2e99aef6e"
format_stamp: "Formatted at 2026-08-08 06:05:31 on dist-test-slave-zj8q"
server_key: "d1687d5a3e2c134de8654da3354eaf58"
server_key_iv: "25ea16b0bf05944ef3320a5403caa533"
server_key_version: "encryptionkey@0"
I20260808 06:05:31.415112 17241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:31.419464 17241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:31.420046 17241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:05:31.420198 17241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:31.420710 17241 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:05:31.421566 17241 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:05:31.421630 17241 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:31.421682 17241 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:05:31.421707 17241 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:31.430323 17241 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.129:39935
I20260808 06:05:31.430351 17445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.129:39935 every 8 connection(s)
I20260808 06:05:31.431150 17241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
I20260808 06:05:31.433265 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 17241
I20260808 06:05:31.433348 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance
I20260808 06:05:31.433652 1630 external_mini_cluster.cc:1428] Setting key fb42577014063967c24f67891f648572
I20260808 06:05:31.435874 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.130:0
--local_ip_for_outbound_sockets=127.1.151.130
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:45285
--builtin_ntp_servers=127.1.151.148:33381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 06:05:31.437453 17211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:31.432842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:59903 (local address 127.1.151.190:45285)
0808 06:05:31.432984 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:31.432987 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:31.434174 (+ 1187us) server_negotiation.cc:407] Connection header received
0808 06:05:31.435732 (+ 1558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:31.435735 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:31.435773 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:31.435831 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:31.436988 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:31.436991 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:31.436993 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:05:31.437006 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:31.437029 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:05:31.437046 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:31.437118 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:31.437304 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 06:05:31.437347 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 06:05:31.438328 17448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:31.433213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:45285 (local address 127.1.151.129:59903)
0808 06:05:31.433985 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:31.434040 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:31.435478 (+ 1438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:31.436022 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:31.436031 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:31.436140 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:31.436682 (+ 542us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 06:05:31.436684 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:31.436734 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 06:05:31.436819 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:31.437122 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:31.437130 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:31.437136 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 06:05:31.437298 (+ 162us) client_negotiation.cc:240] Negotiation successful
0808 06:05:31.437554 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":130,"threads_started":1}
I20260808 06:05:31.439426 17446 heartbeater.cc:347] Connected to a master server at 127.1.151.190:45285
I20260808 06:05:31.439664 17446 heartbeater.cc:464] Registering TS with master...
I20260808 06:05:31.440071 17446 heartbeater.cc:511] Master 127.1.151.190:45285 requested a full tablet report, sending...
I20260808 06:05:31.441566 17151 ts_manager.cc:194] Registered new tserver with Master: dd2740fad39743bb98e346e2e99aef6e (127.1.151.129:39935)
W20260808 06:05:31.547376 17449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:31.547610 17449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:31.547647 17449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:31.551584 17449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:31.551630 17449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:31.551690 17449 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.130
I20260808 06:05:31.556384 17449 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:33381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:45285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.130
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:05:31.557337 17449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:31.558681 17449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:31.567946 17462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:31.568071 17458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:31.567940 17459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:31.567977 17449 server_base.cc:1034] running on GCE node
I20260808 06:05:31.568573 17449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:31.569198 17449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:31.570355 17449 hybrid_clock.cc:648] HybridClock initialized: now 1786169131570338 us; error 27 us; skew 500 ppm
I20260808 06:05:31.572018 17449 webserver.cc:460] Webserver started at http://127.1.151.130:39271/ using document root <none> and password file <none>
I20260808 06:05:31.572572 17449 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:31.572643 17449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:31.572844 17449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:05:31.574873 17449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/data/instance:
uuid: "e9a1eb75b4c84d0f816c1b949e8b3965"
format_stamp: "Formatted at 2026-08-08 06:05:31 on dist-test-slave-zj8q"
server_key: "9b2a6e252e6c7ad7b2d0d46a7cc737a9"
server_key_iv: "81fd292d3e1283ca6b4ac40771420e55"
server_key_version: "encryptionkey@0"
I20260808 06:05:31.575416 17449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance:
uuid: "e9a1eb75b4c84d0f816c1b949e8b3965"
format_stamp: "Formatted at 2026-08-08 06:05:31 on dist-test-slave-zj8q"
server_key: "9b2a6e252e6c7ad7b2d0d46a7cc737a9"
server_key_iv: "81fd292d3e1283ca6b4ac40771420e55"
server_key_version: "encryptionkey@0"
I20260808 06:05:31.578866 17449 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 06:05:31.581216 17470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:31.582376 17449 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:05:31.582479 17449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/wal
uuid: "e9a1eb75b4c84d0f816c1b949e8b3965"
format_stamp: "Formatted at 2026-08-08 06:05:31 on dist-test-slave-zj8q"
server_key: "9b2a6e252e6c7ad7b2d0d46a7cc737a9"
server_key_iv: "81fd292d3e1283ca6b4ac40771420e55"
server_key_version: "encryptionkey@0"
I20260808 06:05:31.582569 17449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:31.586902 17449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:31.587463 17449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:05:31.587611 17449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:31.588116 17449 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:05:31.588964 17449 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:05:31.589010 17449 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:31.589059 17449 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:05:31.589085 17449 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:31.597285 17449 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.130:40915
I20260808 06:05:31.597309 17641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.130:40915 every 8 connection(s)
I20260808 06:05:31.598079 17449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
I20260808 06:05:31.603468 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 17449
I20260808 06:05:31.603570 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance
I20260808 06:05:31.603768 17211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:31.599783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:32927 (local address 127.1.151.190:45285)
0808 06:05:31.599922 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:31.599925 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:31.600898 (+ 973us) server_negotiation.cc:407] Connection header received
0808 06:05:31.602148 (+ 1250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:31.602173 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:31.602222 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:31.602299 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:31.603321 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:31.603325 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:31.603327 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:05:31.603340 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:31.603362 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:05:31.603370 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:31.603448 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:31.603580 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 06:05:31.603625 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 06:05:31.603870 1630 external_mini_cluster.cc:1428] Setting key b100440f044650fd98fafe4056ed1d83
I20260808 06:05:31.604584 17645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:31.600131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:45285 (local address 127.1.151.130:32927)
0808 06:05:31.600756 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:31.600795 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:31.601943 (+ 1148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:31.602464 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:31.602470 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:31.602552 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:31.603055 (+ 503us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 06:05:31.603056 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:31.603095 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 06:05:31.603169 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:31.603441 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:31.603450 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:31.603456 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 06:05:31.603610 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 06:05:31.603833 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":108,"threads_started":1}
I20260808 06:05:31.605674 17642 heartbeater.cc:347] Connected to a master server at 127.1.151.190:45285
I20260808 06:05:31.605751 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.131:0
--local_ip_for_outbound_sockets=127.1.151.131
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:45285
--builtin_ntp_servers=127.1.151.148:33381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 06:05:31.605895 17642 heartbeater.cc:464] Registering TS with master...
I20260808 06:05:31.606297 17642 heartbeater.cc:511] Master 127.1.151.190:45285 requested a full tablet report, sending...
I20260808 06:05:31.607234 17151 ts_manager.cc:194] Registered new tserver with Master: e9a1eb75b4c84d0f816c1b949e8b3965 (127.1.151.130:40915)
W20260808 06:05:31.715308 17646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:31.715552 17646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:31.715590 17646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:31.719529 17646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:31.719585 17646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:31.719640 17646 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.131
I20260808 06:05:31.724607 17646 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:33381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:45285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.131
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:05:31.725666 17646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:31.727049 17646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:31.736320 17656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:31.736331 17652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:31.736361 17646 server_base.cc:1034] running on GCE node
W20260808 06:05:31.736363 17653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:31.736946 17646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:31.737545 17646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:31.738703 17646 hybrid_clock.cc:648] HybridClock initialized: now 1786169131738688 us; error 27 us; skew 500 ppm
I20260808 06:05:31.740429 17646 webserver.cc:460] Webserver started at http://127.1.151.131:38383/ using document root <none> and password file <none>
I20260808 06:05:31.740970 17646 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:31.741030 17646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:31.741209 17646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:05:31.743345 17646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/data/instance:
uuid: "3b1836f87e444c148ed0041f97d92470"
format_stamp: "Formatted at 2026-08-08 06:05:31 on dist-test-slave-zj8q"
server_key: "fd18006f5d6e74ca01c55938c4792ba1"
server_key_iv: "079bc5debc4dd3722bb365cb28935e22"
server_key_version: "encryptionkey@0"
I20260808 06:05:31.743889 17646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance:
uuid: "3b1836f87e444c148ed0041f97d92470"
format_stamp: "Formatted at 2026-08-08 06:05:31 on dist-test-slave-zj8q"
server_key: "fd18006f5d6e74ca01c55938c4792ba1"
server_key_iv: "079bc5debc4dd3722bb365cb28935e22"
server_key_version: "encryptionkey@0"
I20260808 06:05:31.747454 17646 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 06:05:31.749871 17662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:31.751075 17646 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:05:31.751180 17646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/wal
uuid: "3b1836f87e444c148ed0041f97d92470"
format_stamp: "Formatted at 2026-08-08 06:05:31 on dist-test-slave-zj8q"
server_key: "fd18006f5d6e74ca01c55938c4792ba1"
server_key_iv: "079bc5debc4dd3722bb365cb28935e22"
server_key_version: "encryptionkey@0"
I20260808 06:05:31.751284 17646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:31.755676 17646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:31.756279 17646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:05:31.756436 17646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:31.756984 17646 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:05:31.757889 17646 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:05:31.757936 17646 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:31.757978 17646 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:05:31.758008 17646 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:31.766536 17646 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.131:34075
I20260808 06:05:31.766572 17829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.131:34075 every 8 connection(s)
I20260808 06:05:31.767393 17646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
I20260808 06:05:31.772142 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 17646
I20260808 06:05:31.772246 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance
I20260808 06:05:31.772535 1630 external_mini_cluster.cc:1428] Setting key d7322a4577445ee02bef7312ee53018b
I20260808 06:05:31.773273 17211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:31.769160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:48839 (local address 127.1.151.190:45285)
0808 06:05:31.769279 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:31.769283 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:31.770363 (+ 1080us) server_negotiation.cc:407] Connection header received
0808 06:05:31.771647 (+ 1284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:31.771650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:31.771687 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:31.771746 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:31.772833 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:31.772837 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:31.772840 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:05:31.772854 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:31.772876 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:05:31.772884 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:31.772963 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:31.773089 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 06:05:31.773135 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 06:05:31.774087 17833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:31.769527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:45285 (local address 127.1.151.131:48839)
0808 06:05:31.770218 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:31.770261 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:31.771449 (+ 1188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:31.771894 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:31.771901 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:31.771985 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:31.772551 (+ 566us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 06:05:31.772552 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:31.772602 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 06:05:31.772684 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:31.772955 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:31.772963 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:31.772969 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 06:05:31.773125 (+ 156us) client_negotiation.cc:240] Negotiation successful
0808 06:05:31.773373 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":114,"threads_started":1}
I20260808 06:05:31.775089 17831 heartbeater.cc:347] Connected to a master server at 127.1.151.190:45285
I20260808 06:05:31.775285 17831 heartbeater.cc:464] Registering TS with master...
I20260808 06:05:31.775640 17831 heartbeater.cc:511] Master 127.1.151.190:45285 requested a full tablet report, sending...
I20260808 06:05:31.776526 17151 ts_manager.cc:194] Registered new tserver with Master: 3b1836f87e444c148ed0041f97d92470 (127.1.151.131:34075)
I20260808 06:05:31.786177 1630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 06:05:31.788954 17211 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:05:31.788108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36846 (local address 127.1.151.190:45285)
0808 06:05:31.788240 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:31.788245 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:31.788421 (+ 176us) server_negotiation.cc:407] Connection header received
0808 06:05:31.788492 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:31.788495 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:31.788528 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:31.788606 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:31.788829 (+ 223us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36846: BlockingRecv error: recv got EOF from 127.0.0.1:36846 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 06:05:31.789227 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 17241
I20260808 06:05:31.821766 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 17449
I20260808 06:05:31.833338 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 17646
I20260808 06:05:31.853816 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 17107
2026-08-08T06:05:31Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (848 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:05:31 dist-test-slave-zj8q krb5kdc[17844](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:05:31 dist-test-slave-zj8q krb5kdc[17844](info): set up 2 sockets
Aug 08 06:05:31 dist-test-slave-zj8q krb5kdc[17844](info): commencing operation
krb5kdc: starting...
W20260808 06:05:33.001708 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:05:33 dist-test-slave-zj8q krb5kdc[17844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:05:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:05:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.151.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
I20260808 06:05:33.121100 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:32893
--webserver_interface=127.1.151.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:44175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:32893
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:05:33.230873 17864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:33.231114 17864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:33.231169 17864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:33.235039 17864 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:05:33.235090 17864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:33.235116 17864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:33.235137 17864 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:05:33.235167 17864 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:05:33.240159 17864 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:44175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:32893
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:32893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17864
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:05:33.241101 17864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:33.242067 17864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:33.250342 17873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:33.250407 17864 server_base.cc:1034] running on GCE node
W20260808 06:05:33.250300 17871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:33.250300 17870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:33.251180 17864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:33.252249 17864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:33.253422 17864 hybrid_clock.cc:648] HybridClock initialized: now 1786169133253407 us; error 28 us; skew 500 ppm
Aug 08 06:05:33 dist-test-slave-zj8q krb5kdc[17844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169133, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:33.256168 17864 init.cc:377] Logged in from keytab as kudu/127.1.151.190@KRBTEST.COM (short username kudu)
I20260808 06:05:33.257067 17864 webserver.cc:460] Webserver started at http://127.1.151.190:34279/ using document root <none> and password file <none>
I20260808 06:05:33.257603 17864 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:33.257670 17864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:33.257871 17864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:05:33.259938 17864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "09912ce896fc46fb9474d451d2b52d4f"
format_stamp: "Formatted at 2026-08-08 06:05:33 on dist-test-slave-zj8q"
server_key: "a4d1511f6d6b8f6a3a58e4946e806b8d"
server_key_iv: "c790ee3919356e7ddfb2e3aaa301508e"
server_key_version: "encryptionkey@0"
I20260808 06:05:33.260502 17864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "09912ce896fc46fb9474d451d2b52d4f"
format_stamp: "Formatted at 2026-08-08 06:05:33 on dist-test-slave-zj8q"
server_key: "a4d1511f6d6b8f6a3a58e4946e806b8d"
server_key_iv: "c790ee3919356e7ddfb2e3aaa301508e"
server_key_version: "encryptionkey@0"
I20260808 06:05:33.264269 17864 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260808 06:05:33.266731 17881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:33.267885 17864 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:05:33.268002 17864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "09912ce896fc46fb9474d451d2b52d4f"
format_stamp: "Formatted at 2026-08-08 06:05:33 on dist-test-slave-zj8q"
server_key: "a4d1511f6d6b8f6a3a58e4946e806b8d"
server_key_iv: "c790ee3919356e7ddfb2e3aaa301508e"
server_key_version: "encryptionkey@0"
I20260808 06:05:33.268111 17864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:33.291023 17864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:33.294063 17864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:05:33.294268 17864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:33.301433 17864 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:32893
I20260808 06:05:33.301693 17968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:32893 every 8 connection(s)
I20260808 06:05:33.302470 17864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:05:33.305147 17969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:05:33.306968 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 17864
I20260808 06:05:33.307096 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:05:33.307403 1630 external_mini_cluster.cc:1428] Setting key 8efb7b354741a5401072cebe44aa41a7
I20260808 06:05:33.311597 17969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09912ce896fc46fb9474d451d2b52d4f: Bootstrap starting.
I20260808 06:05:33.313736 17969 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09912ce896fc46fb9474d451d2b52d4f: Neither blocks nor log segments found. Creating new log.
I20260808 06:05:33.314468 17969 log.cc:824] T 00000000000000000000000000000000 P 09912ce896fc46fb9474d451d2b52d4f: Log is configured to *not* fsync() on all Append() calls
Aug 08 06:05:33 dist-test-slave-zj8q krb5kdc[17844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:05:33.316412 17969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09912ce896fc46fb9474d451d2b52d4f: No bootstrap required, opened a new log
I20260808 06:05:33.320626 17969 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09912ce896fc46fb9474d451d2b52d4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09912ce896fc46fb9474d451d2b52d4f" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 32893 } }
I20260808 06:05:33.320888 17969 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09912ce896fc46fb9474d451d2b52d4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:05:33.320955 17969 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09912ce896fc46fb9474d451d2b52d4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09912ce896fc46fb9474d451d2b52d4f, State: Initialized, Role: FOLLOWER
I20260808 06:05:33.321415 17969 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09912ce896fc46fb9474d451d2b52d4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09912ce896fc46fb9474d451d2b52d4f" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 32893 } }
I20260808 06:05:33.321556 17974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:33.308862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42274 (local address 127.1.151.190:32893)
0808 06:05:33.309290 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:33.309305 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:33.309340 (+ 35us) server_negotiation.cc:407] Connection header received
0808 06:05:33.310086 (+ 746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:33.310111 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:33.310463 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:33.310761 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:33.311574 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:33.312586 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:33.313421 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:33.313926 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:33.314082 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:33.316603 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:33.316609 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:33.316624 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:33.316655 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:33.318534 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:33.318972 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:33.318978 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:33.318983 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:33.319122 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:33.319351 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:33.319354 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:33.319355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:33.319839 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:33.320046 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:33.320303 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 06:05:33.320583 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":124,"threads_started":1}
I20260808 06:05:33.321579 17969 raft_consensus.cc:388] T 00000000000000000000000000000000 P 09912ce896fc46fb9474d451d2b52d4f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:05:33.321646 17969 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09912ce896fc46fb9474d451d2b52d4f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:05:33.321728 17969 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09912ce896fc46fb9474d451d2b52d4f [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:05:33.322576 17969 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09912ce896fc46fb9474d451d2b52d4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09912ce896fc46fb9474d451d2b52d4f" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 32893 } }
I20260808 06:05:33.322933 17969 leader_election.cc:302] T 00000000000000000000000000000000 P 09912ce896fc46fb9474d451d2b52d4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 09912ce896fc46fb9474d451d2b52d4f; no voters:
I20260808 06:05:33.323259 17969 leader_election.cc:288] T 00000000000000000000000000000000 P 09912ce896fc46fb9474d451d2b52d4f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:05:33.323359 17977 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09912ce896fc46fb9474d451d2b52d4f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:05:33.323570 17977 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09912ce896fc46fb9474d451d2b52d4f [term 1 LEADER]: Becoming Leader. State: Replica: 09912ce896fc46fb9474d451d2b52d4f, State: Running, Role: LEADER
I20260808 06:05:33.323861 17977 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09912ce896fc46fb9474d451d2b52d4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09912ce896fc46fb9474d451d2b52d4f" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 32893 } }
I20260808 06:05:33.324393 17969 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09912ce896fc46fb9474d451d2b52d4f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:05:33.324954 17977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09912ce896fc46fb9474d451d2b52d4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09912ce896fc46fb9474d451d2b52d4f. Latest consensus state: current_term: 1 leader_uuid: "09912ce896fc46fb9474d451d2b52d4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09912ce896fc46fb9474d451d2b52d4f" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 32893 } } }
I20260808 06:05:33.325067 17977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09912ce896fc46fb9474d451d2b52d4f [sys.catalog]: This master's current role is: LEADER
I20260808 06:05:33.325596 17985 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:05:33.325736 17978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09912ce896fc46fb9474d451d2b52d4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "09912ce896fc46fb9474d451d2b52d4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09912ce896fc46fb9474d451d2b52d4f" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 32893 } } }
I20260808 06:05:33.325886 17978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09912ce896fc46fb9474d451d2b52d4f [sys.catalog]: This master's current role is: LEADER
I20260808 06:05:33.328472 17985 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:05:33.333644 17985 catalog_manager.cc:1347] Generated new cluster ID: 176680d3bb984656ba40a57d4daf298d
I20260808 06:05:33.333753 17985 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:05:33.345971 17985 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:05:33.346858 17985 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:05:33.352917 17985 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09912ce896fc46fb9474d451d2b52d4f: Generated new TSK 0
I20260808 06:05:33.353502 17985 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.151.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab.
Entry for principal kudu/127.1.151.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab.
I20260808 06:05:33.385016 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.129:0
--local_ip_for_outbound_sockets=127.1.151.129
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:32893
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:44175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:05:33.531044 18011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:33.531288 18011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:33.531330 18011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:33.535177 18011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:33.535228 18011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:33.535282 18011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.129
I20260808 06:05:33.540300 18011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:44175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:32893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.129
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:05:33.541275 18011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:33.542248 18011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:33.551699 18023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:33.551844 18011 server_base.cc:1034] running on GCE node
W20260808 06:05:33.551692 18020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:33.551761 18021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:33.552395 18011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:33.553010 18011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:33.554198 18011 hybrid_clock.cc:648] HybridClock initialized: now 1786169133554184 us; error 41 us; skew 500 ppm
Aug 08 06:05:33 dist-test-slave-zj8q krb5kdc[17844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169133, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:33.556777 18011 init.cc:377] Logged in from keytab as kudu/127.1.151.129@KRBTEST.COM (short username kudu)
I20260808 06:05:33.557651 18011 webserver.cc:460] Webserver started at http://127.1.151.129:42469/ using document root <none> and password file <none>
I20260808 06:05:33.558257 18011 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:33.558331 18011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:33.558531 18011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:05:33.560496 18011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/data/instance:
uuid: "17007b4f9a854b96a5f9bb31814441ff"
format_stamp: "Formatted at 2026-08-08 06:05:33 on dist-test-slave-zj8q"
server_key: "d7642af66236dcfdda89214d414a9284"
server_key_iv: "47fc321b3b4634a1bf3b8b770050c117"
server_key_version: "encryptionkey@0"
I20260808 06:05:33.561134 18011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance:
uuid: "17007b4f9a854b96a5f9bb31814441ff"
format_stamp: "Formatted at 2026-08-08 06:05:33 on dist-test-slave-zj8q"
server_key: "d7642af66236dcfdda89214d414a9284"
server_key_iv: "47fc321b3b4634a1bf3b8b770050c117"
server_key_version: "encryptionkey@0"
I20260808 06:05:33.564841 18011 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 06:05:33.567255 18032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:33.568459 18011 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 06:05:33.568558 18011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/wal
uuid: "17007b4f9a854b96a5f9bb31814441ff"
format_stamp: "Formatted at 2026-08-08 06:05:33 on dist-test-slave-zj8q"
server_key: "d7642af66236dcfdda89214d414a9284"
server_key_iv: "47fc321b3b4634a1bf3b8b770050c117"
server_key_version: "encryptionkey@0"
I20260808 06:05:33.568657 18011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:33.582206 18011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:33.584771 18011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:05:33.584932 18011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:33.585489 18011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:05:33.586349 18011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:05:33.586404 18011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:33.586448 18011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:05:33.586476 18011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:33.594907 18011 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.129:41359
I20260808 06:05:33.594971 18213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.129:41359 every 8 connection(s)
I20260808 06:05:33.595737 18011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
I20260808 06:05:33.602013 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 18011
I20260808 06:05:33.602115 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance
I20260808 06:05:33.602392 1630 external_mini_cluster.cc:1428] Setting key fd4e00dc481cf6d7f0a30b676b60b8ae
Aug 08 06:05:33 dist-test-slave-zj8q krb5kdc[17844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169133, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:05:33.609409 17974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:33.597404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:55471 (local address 127.1.151.190:32893)
0808 06:05:33.597532 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:33.597535 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:33.598309 (+ 774us) server_negotiation.cc:407] Connection header received
0808 06:05:33.599670 (+ 1361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:33.599673 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:33.599735 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:33.599809 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:33.600963 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:33.601508 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:33.602739 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:33.603025 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:33.603127 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:33.605806 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:33.605810 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:33.605813 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:33.605843 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:33.607362 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:33.607893 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:33.607897 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:33.607898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:33.607989 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:33.608288 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:33.608291 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:33.608292 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:33.608512 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:33.608600 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:33.609099 (+ 499us) server_negotiation.cc:299] Negotiation successful
0808 06:05:33.609240 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 06:05:33.610205 18218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:33.597720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:32893 (local address 127.1.151.129:55471)
0808 06:05:33.598172 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:33.598221 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:33.599456 (+ 1235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:33.599993 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:33.600000 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:33.600352 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:33.600815 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:33.600829 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:33.601628 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:33.601631 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:33.602614 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:33.602625 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:33.603169 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:33.603173 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:33.603335 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:33.603951 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:33.603971 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:33.605619 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:33.607492 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:33.607497 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:33.607509 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:33.607793 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:33.608070 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:33.608072 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:33.608074 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:33.608209 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:33.608601 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:33.608605 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:33.608863 (+ 258us) client_negotiation.cc:769] Sending connection context
0808 06:05:33.609016 (+ 153us) client_negotiation.cc:240] Negotiation successful
0808 06:05:33.609321 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":116,"threads_started":1}
I20260808 06:05:33.611676 18216 heartbeater.cc:347] Connected to a master server at 127.1.151.190:32893
I20260808 06:05:33.611905 18216 heartbeater.cc:464] Registering TS with master...
I20260808 06:05:33.612385 18216 heartbeater.cc:511] Master 127.1.151.190:32893 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.151.130@KRBTEST.COM; defaulting to no policy
I20260808 06:05:33.613976 17915 ts_manager.cc:194] Registered new tserver with Master: 17007b4f9a854b96a5f9bb31814441ff (127.1.151.129:41359)
I20260808 06:05:33.615139 17915 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.129@KRBTEST.COM'} at 127.1.151.129:55471
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab.
Entry for principal kudu/127.1.151.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab.
I20260808 06:05:33.631006 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.130:0
--local_ip_for_outbound_sockets=127.1.151.130
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:32893
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:44175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:05:33.738729 18223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:33.738978 18223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:33.739020 18223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:33.742916 18223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:33.742966 18223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:33.743031 18223 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.130
I20260808 06:05:33.748642 18223 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:44175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:32893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18223
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.130
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:05:33.749581 18223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:33.750576 18223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:33.759750 18235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:33.759769 18239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:33.759838 18223 server_base.cc:1034] running on GCE node
W20260808 06:05:33.759788 18236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:33.760322 18223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:33.760934 18223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:33.762109 18223 hybrid_clock.cc:648] HybridClock initialized: now 1786169133762078 us; error 40 us; skew 500 ppm
Aug 08 06:05:33 dist-test-slave-zj8q krb5kdc[17844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169133, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:33.764827 18223 init.cc:377] Logged in from keytab as kudu/127.1.151.130@KRBTEST.COM (short username kudu)
I20260808 06:05:33.765645 18223 webserver.cc:460] Webserver started at http://127.1.151.130:45223/ using document root <none> and password file <none>
I20260808 06:05:33.766270 18223 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:33.766360 18223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:33.766566 18223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:05:33.768501 18223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/data/instance:
uuid: "a72b3920c721481e90c181aa5bc059ee"
format_stamp: "Formatted at 2026-08-08 06:05:33 on dist-test-slave-zj8q"
server_key: "02bdc14e209b192b51c188e62b149381"
server_key_iv: "37111ad777ba0932cb4e57ae9d2e083b"
server_key_version: "encryptionkey@0"
I20260808 06:05:33.769042 18223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance:
uuid: "a72b3920c721481e90c181aa5bc059ee"
format_stamp: "Formatted at 2026-08-08 06:05:33 on dist-test-slave-zj8q"
server_key: "02bdc14e209b192b51c188e62b149381"
server_key_iv: "37111ad777ba0932cb4e57ae9d2e083b"
server_key_version: "encryptionkey@0"
I20260808 06:05:33.772835 18223 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 06:05:33.775094 18248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:33.776254 18223 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:05:33.776360 18223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/wal
uuid: "a72b3920c721481e90c181aa5bc059ee"
format_stamp: "Formatted at 2026-08-08 06:05:33 on dist-test-slave-zj8q"
server_key: "02bdc14e209b192b51c188e62b149381"
server_key_iv: "37111ad777ba0932cb4e57ae9d2e083b"
server_key_version: "encryptionkey@0"
I20260808 06:05:33.776448 18223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:33.791326 18223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:33.796928 18223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:05:33.797108 18223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:33.797683 18223 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:05:33.798532 18223 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:05:33.798581 18223 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:33.798635 18223 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:05:33.798671 18223 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:33.806970 18223 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.130:46841
I20260808 06:05:33.807024 18427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.130:46841 every 8 connection(s)
I20260808 06:05:33.807832 18223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
I20260808 06:05:33.816836 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 18223
I20260808 06:05:33.816929 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance
Aug 08 06:05:33 dist-test-slave-zj8q krb5kdc[17844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169133, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:05:33.817209 1630 external_mini_cluster.cc:1428] Setting key 2897eb640ab133017beba2cc013eb9ab
I20260808 06:05:33.820958 17974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:33.809470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:56683 (local address 127.1.151.190:32893)
0808 06:05:33.809601 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:33.809604 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:33.810453 (+ 849us) server_negotiation.cc:407] Connection header received
0808 06:05:33.811800 (+ 1347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:33.811803 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:33.811865 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:33.811937 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:33.813062 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:33.813590 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:33.814777 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:33.815003 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:33.815094 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:33.817537 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:33.817541 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:33.817544 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:33.817573 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:33.819003 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:33.819489 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:33.819493 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:33.819495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:33.819589 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:33.819883 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:33.819886 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:33.819887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:33.820096 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:33.820201 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:33.820664 (+ 463us) server_negotiation.cc:299] Negotiation successful
0808 06:05:33.820795 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 06:05:33.821844 18432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:33.809852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:32893 (local address 127.1.151.130:56683)
0808 06:05:33.810313 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:33.810356 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:33.811588 (+ 1232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:33.812117 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:33.812124 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:33.812471 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:33.812917 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:33.812930 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:33.813712 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:33.813715 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:33.814659 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:33.814669 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:33.815149 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:33.815175 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:33.815325 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:33.815864 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:33.815880 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:33.817372 (+ 1492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:33.819136 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:33.819140 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:33.819151 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:33.819395 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:33.819680 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:33.819682 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:33.819683 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:33.819808 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:33.820190 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:33.820195 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:33.820407 (+ 212us) client_negotiation.cc:769] Sending connection context
0808 06:05:33.820590 (+ 183us) client_negotiation.cc:240] Negotiation successful
0808 06:05:33.820886 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":116,"threads_started":1}
I20260808 06:05:33.822984 18428 heartbeater.cc:347] Connected to a master server at 127.1.151.190:32893
I20260808 06:05:33.823211 18428 heartbeater.cc:464] Registering TS with master...
I20260808 06:05:33.823659 18428 heartbeater.cc:511] Master 127.1.151.190:32893 requested a full tablet report, sending...
I20260808 06:05:33.824620 17915 ts_manager.cc:194] Registered new tserver with Master: a72b3920c721481e90c181aa5bc059ee (127.1.151.130:46841)
I20260808 06:05:33.825156 17915 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.130@KRBTEST.COM'} at 127.1.151.130:56683
WARNING: no policy specified for kudu/127.1.151.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab.
Entry for principal kudu/127.1.151.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab.
I20260808 06:05:33.844834 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.131:0
--local_ip_for_outbound_sockets=127.1.151.131
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:32893
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:44175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:05:33.954239 18436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:05:33.954480 18436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:05:33.954522 18436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:05:33.958824 18436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:05:33.958884 18436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:05:33.958940 18436 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.131
I20260808 06:05:33.964048 18436 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:44175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:32893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.131
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:05:33.964957 18436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:05:33.965921 18436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:05:33.977052 18448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:33.977053 18446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:05:33.978535 18451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:05:33.978858 18436 server_base.cc:1034] running on GCE node
I20260808 06:05:33.979354 18436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:05:33.980116 18436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:05:33.981310 18436 hybrid_clock.cc:648] HybridClock initialized: now 1786169133981281 us; error 40 us; skew 500 ppm
Aug 08 06:05:33 dist-test-slave-zj8q krb5kdc[17844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169133, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:05:33.983914 18436 init.cc:377] Logged in from keytab as kudu/127.1.151.131@KRBTEST.COM (short username kudu)
I20260808 06:05:33.984712 18436 webserver.cc:460] Webserver started at http://127.1.151.131:37939/ using document root <none> and password file <none>
I20260808 06:05:33.985246 18436 fs_manager.cc:359] Metadata directory not provided
I20260808 06:05:33.985313 18436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:05:33.985498 18436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:05:33.987540 18436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/data/instance:
uuid: "d0f640df97f540d5878c0e44f839cba2"
format_stamp: "Formatted at 2026-08-08 06:05:33 on dist-test-slave-zj8q"
server_key: "3602c783c9cae99c1e45fd4cbd108176"
server_key_iv: "7888a01a57dc9af3611e90471ef02ed2"
server_key_version: "encryptionkey@0"
I20260808 06:05:33.988090 18436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance:
uuid: "d0f640df97f540d5878c0e44f839cba2"
format_stamp: "Formatted at 2026-08-08 06:05:33 on dist-test-slave-zj8q"
server_key: "3602c783c9cae99c1e45fd4cbd108176"
server_key_iv: "7888a01a57dc9af3611e90471ef02ed2"
server_key_version: "encryptionkey@0"
I20260808 06:05:33.991545 18436 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 06:05:33.993762 18464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:33.994900 18436 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:05:33.995007 18436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/wal
uuid: "d0f640df97f540d5878c0e44f839cba2"
format_stamp: "Formatted at 2026-08-08 06:05:33 on dist-test-slave-zj8q"
server_key: "3602c783c9cae99c1e45fd4cbd108176"
server_key_iv: "7888a01a57dc9af3611e90471ef02ed2"
server_key_version: "encryptionkey@0"
I20260808 06:05:33.995095 18436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:05:34.011318 18436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:05:34.014245 18436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:05:34.014410 18436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:05:34.014973 18436 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:05:34.015930 18436 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:05:34.015987 18436 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:34.016043 18436 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:05:34.016072 18436 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:05:34.025837 18436 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.131:36419
I20260808 06:05:34.025873 18664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.131:36419 every 8 connection(s)
I20260808 06:05:34.026759 18436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
I20260808 06:05:34.031073 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 18436
I20260808 06:05:34.031196 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance
I20260808 06:05:34.031515 1630 external_mini_cluster.cc:1428] Setting key 1c28eda9e3e0c3b6346fd766973aab5c
Aug 08 06:05:34 dist-test-slave-zj8q krb5kdc[17844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169133, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:05:34.040184 17974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:34.028447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:38361 (local address 127.1.151.190:32893)
0808 06:05:34.028607 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:34.028613 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:34.029414 (+ 801us) server_negotiation.cc:407] Connection header received
0808 06:05:34.030802 (+ 1388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:34.030805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:34.030855 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:34.030918 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:34.032370 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:34.033147 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:34.034433 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:34.034703 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:34.034781 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:34.036960 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:34.036963 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:34.036965 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:34.036988 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:34.038426 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:34.038865 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:34.038868 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:34.038869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:34.038958 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:34.039266 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:34.039269 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:34.039271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:34.039456 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:34.039539 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:34.039960 (+ 421us) server_negotiation.cc:299] Negotiation successful
0808 06:05:34.040065 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 06:05:34.040891 18668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:34.028851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:32893 (local address 127.1.151.131:38361)
0808 06:05:34.029275 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:34.029315 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:34.030590 (+ 1275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:34.031144 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:34.031153 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:34.031627 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:05:34.032213 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:34.032232 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:34.033284 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:34.033287 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:34.034327 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:34.034338 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:34.034819 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:34.034821 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:34.034953 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:34.035458 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:05:34.035473 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:05:34.036806 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:05:34.038526 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:34.038530 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:34.038541 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:34.038780 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:34.039026 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:05:34.039028 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:05:34.039029 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:05:34.039164 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:05:34.039549 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:05:34.039554 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:05:34.039771 (+ 217us) client_negotiation.cc:769] Sending connection context
0808 06:05:34.039917 (+ 146us) client_negotiation.cc:240] Negotiation successful
0808 06:05:34.040162 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":109,"threads_started":1}
I20260808 06:05:34.041903 18665 heartbeater.cc:347] Connected to a master server at 127.1.151.190:32893
I20260808 06:05:34.042120 18665 heartbeater.cc:464] Registering TS with master...
I20260808 06:05:34.042593 18665 heartbeater.cc:511] Master 127.1.151.190:32893 requested a full tablet report, sending...
I20260808 06:05:34.043570 17915 ts_manager.cc:194] Registered new tserver with Master: d0f640df97f540d5878c0e44f839cba2 (127.1.151.131:36419)
I20260808 06:05:34.044209 17915 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.131@KRBTEST.COM'} at 127.1.151.131:38361
I20260808 06:05:34.045323 1630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 06:05:34.053862 17974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:34.047226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42278 (local address 127.1.151.190:32893)
0808 06:05:34.047342 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:34.047346 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:34.047509 (+ 163us) server_negotiation.cc:407] Connection header received
0808 06:05:34.047625 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:34.047628 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:34.047671 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:34.047733 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:05:34.048402 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:34.048881 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:34.049693 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:34.049904 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:34.049988 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:34.051250 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:05:34.051253 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:05:34.051256 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:05:34.051279 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:05:34.052339 (+ 1060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:34.052729 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:34.052733 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:34.052735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:34.052810 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:05:34.053052 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:05:34.053055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:05:34.053057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:05:34.053269 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:05:34.053388 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:34.053618 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 06:05:34.053721 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 06:05:34.056404 17915 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42278:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 06:05:34.059604 17915 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 06:05:34.074131 18677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:34.067810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:41359 (local address 127.0.0.1:47966)
0808 06:05:34.068124 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:34.068170 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:34.068291 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:34.069428 (+ 1137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:34.069434 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:34.069455 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:05:34.069663 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:34.069673 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:34.071074 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:34.071078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:34.072524 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:34.072536 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:34.073609 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:34.073613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:34.073716 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:34.073759 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 06:05:34.073861 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 06:05:34.073976 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":80,"threads_started":1}
I20260808 06:05:34.074393 18684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:34.068479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43982 (local address 127.1.151.130:46841)
0808 06:05:34.069647 (+ 1168us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:34.069660 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:34.069684 (+ 24us) server_negotiation.cc:407] Connection header received
0808 06:05:34.069771 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:34.069777 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:34.069962 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:34.070145 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:05:34.070451 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:34.071421 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:34.072969 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:34.073777 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:34.073879 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:34.074010 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:34.074129 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 06:05:34.074265 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1055,"thread_start_us":149,"threads_started":1}
I20260808 06:05:34.074344 18678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:34.067916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:46841 (local address 127.0.0.1:43982)
0808 06:05:34.068885 (+ 969us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:34.068898 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:34.068983 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:34.070140 (+ 1157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:34.070144 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:34.070190 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:05:34.070301 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:34.070307 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:34.071658 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:34.071662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:34.072853 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:34.072862 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:34.073896 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:34.073900 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:34.073999 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:34.074031 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 06:05:34.074102 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 06:05:34.074210 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":907,"mutex_wait_us":34,"thread_start_us":87,"threads_started":1}
I20260808 06:05:34.074678 18679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:34.068290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47966 (local address 127.1.151.129:41359)
0808 06:05:34.068669 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:34.068685 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:34.068708 (+ 23us) server_negotiation.cc:407] Connection header received
0808 06:05:34.068787 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:34.068794 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:34.068995 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:34.069204 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:05:34.069806 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:34.070939 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:34.072664 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:34.073471 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:34.073580 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:34.073725 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:34.074446 (+ 721us) server_negotiation.cc:299] Negotiation successful
0808 06:05:34.074542 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":81,"threads_started":1}
I20260808 06:05:34.075304 18681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:34.068505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:36419 (local address 127.0.0.1:47562)
0808 06:05:34.069281 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:34.069295 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:34.069388 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:34.070385 (+ 997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:34.070388 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:34.070408 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:05:34.070502 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:34.070509 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:34.071765 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:34.071769 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:34.073681 (+ 1912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:34.073690 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:34.074917 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:34.074920 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:34.075020 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:34.075042 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 06:05:34.075125 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 06:05:34.075189 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"thread_start_us":102,"threads_started":1}
I20260808 06:05:34.075438 18685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:34.068675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47562 (local address 127.1.151.131:36419)
0808 06:05:34.069945 (+ 1270us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:34.069956 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:34.069977 (+ 21us) server_negotiation.cc:407] Connection header received
0808 06:05:34.070045 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:34.070051 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:34.070249 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:34.070396 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:05:34.070609 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:34.071616 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:34.073811 (+ 2195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:34.074609 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:34.074702 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:34.074839 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:34.075177 (+ 338us) server_negotiation.cc:299] Negotiation successful
0808 06:05:34.075276 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1173,"thread_start_us":154,"threads_started":1}
I20260808 06:05:34.077272 18327 tablet_service.cc:1467] Processing CreateTablet for tablet 779fcf76782148ce9581af51fd87eca2 (DEFAULT_TABLE table=test-table [id=2efb9f4989834581936d5fe8d5ab9b0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:05:34.077702 18545 tablet_service.cc:1467] Processing CreateTablet for tablet 779fcf76782148ce9581af51fd87eca2 (DEFAULT_TABLE table=test-table [id=2efb9f4989834581936d5fe8d5ab9b0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:05:34.078203 18327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 779fcf76782148ce9581af51fd87eca2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:05:34.078650 18545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 779fcf76782148ce9581af51fd87eca2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:05:34.081139 18104 tablet_service.cc:1467] Processing CreateTablet for tablet 779fcf76782148ce9581af51fd87eca2 (DEFAULT_TABLE table=test-table [id=2efb9f4989834581936d5fe8d5ab9b0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:05:34.082120 18104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 779fcf76782148ce9581af51fd87eca2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:05:34.084904 18686 tablet_bootstrap.cc:492] T 779fcf76782148ce9581af51fd87eca2 P d0f640df97f540d5878c0e44f839cba2: Bootstrap starting.
I20260808 06:05:34.086279 18688 tablet_bootstrap.cc:492] T 779fcf76782148ce9581af51fd87eca2 P a72b3920c721481e90c181aa5bc059ee: Bootstrap starting.
I20260808 06:05:34.087092 18686 tablet_bootstrap.cc:654] T 779fcf76782148ce9581af51fd87eca2 P d0f640df97f540d5878c0e44f839cba2: Neither blocks nor log segments found. Creating new log.
I20260808 06:05:34.088158 18686 log.cc:824] T 779fcf76782148ce9581af51fd87eca2 P d0f640df97f540d5878c0e44f839cba2: Log is configured to *not* fsync() on all Append() calls
I20260808 06:05:34.088871 18688 tablet_bootstrap.cc:654] T 779fcf76782148ce9581af51fd87eca2 P a72b3920c721481e90c181aa5bc059ee: Neither blocks nor log segments found. Creating new log.
I20260808 06:05:34.089709 18688 log.cc:824] T 779fcf76782148ce9581af51fd87eca2 P a72b3920c721481e90c181aa5bc059ee: Log is configured to *not* fsync() on all Append() calls
I20260808 06:05:34.089943 18690 tablet_bootstrap.cc:492] T 779fcf76782148ce9581af51fd87eca2 P 17007b4f9a854b96a5f9bb31814441ff: Bootstrap starting.
I20260808 06:05:34.090173 18686 tablet_bootstrap.cc:492] T 779fcf76782148ce9581af51fd87eca2 P d0f640df97f540d5878c0e44f839cba2: No bootstrap required, opened a new log
I20260808 06:05:34.090361 18686 ts_tablet_manager.cc:1397] T 779fcf76782148ce9581af51fd87eca2 P d0f640df97f540d5878c0e44f839cba2: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 06:05:34.091677 18690 tablet_bootstrap.cc:654] T 779fcf76782148ce9581af51fd87eca2 P 17007b4f9a854b96a5f9bb31814441ff: Neither blocks nor log segments found. Creating new log.
I20260808 06:05:34.091689 18688 tablet_bootstrap.cc:492] T 779fcf76782148ce9581af51fd87eca2 P a72b3920c721481e90c181aa5bc059ee: No bootstrap required, opened a new log
I20260808 06:05:34.091857 18688 ts_tablet_manager.cc:1397] T 779fcf76782148ce9581af51fd87eca2 P a72b3920c721481e90c181aa5bc059ee: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 06:05:34.092610 18690 log.cc:824] T 779fcf76782148ce9581af51fd87eca2 P 17007b4f9a854b96a5f9bb31814441ff: Log is configured to *not* fsync() on all Append() calls
I20260808 06:05:34.094102 18690 tablet_bootstrap.cc:492] T 779fcf76782148ce9581af51fd87eca2 P 17007b4f9a854b96a5f9bb31814441ff: No bootstrap required, opened a new log
I20260808 06:05:34.094305 18690 ts_tablet_manager.cc:1397] T 779fcf76782148ce9581af51fd87eca2 P 17007b4f9a854b96a5f9bb31814441ff: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 06:05:34.094142 18686 raft_consensus.cc:348] T 779fcf76782148ce9581af51fd87eca2 P d0f640df97f540d5878c0e44f839cba2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17007b4f9a854b96a5f9bb31814441ff" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 41359 } } peers { permanent_uuid: "a72b3920c721481e90c181aa5bc059ee" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 46841 } } peers { permanent_uuid: "d0f640df97f540d5878c0e44f839cba2" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 36419 } }
I20260808 06:05:34.094429 18686 raft_consensus.cc:374] T 779fcf76782148ce9581af51fd87eca2 P d0f640df97f540d5878c0e44f839cba2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:05:34.094484 18686 raft_consensus.cc:729] T 779fcf76782148ce9581af51fd87eca2 P d0f640df97f540d5878c0e44f839cba2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f640df97f540d5878c0e44f839cba2, State: Initialized, Role: FOLLOWER
I20260808 06:05:34.094925 18686 consensus_queue.cc:260] T 779fcf76782148ce9581af51fd87eca2 P d0f640df97f540d5878c0e44f839cba2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17007b4f9a854b96a5f9bb31814441ff" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 41359 } } peers { permanent_uuid: "a72b3920c721481e90c181aa5bc059ee" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 46841 } } peers { permanent_uuid: "d0f640df97f540d5878c0e44f839cba2" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 36419 } }
I20260808 06:05:34.095606 18665 heartbeater.cc:503] Master 127.1.151.190:32893 was elected leader, sending a full tablet report...
I20260808 06:05:34.095497 18688 raft_consensus.cc:348] T 779fcf76782148ce9581af51fd87eca2 P a72b3920c721481e90c181aa5bc059ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17007b4f9a854b96a5f9bb31814441ff" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 41359 } } peers { permanent_uuid: "a72b3920c721481e90c181aa5bc059ee" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 46841 } } peers { permanent_uuid: "d0f640df97f540d5878c0e44f839cba2" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 36419 } }
I20260808 06:05:34.095736 18688 raft_consensus.cc:374] T 779fcf76782148ce9581af51fd87eca2 P a72b3920c721481e90c181aa5bc059ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:05:34.095785 18688 raft_consensus.cc:729] T 779fcf76782148ce9581af51fd87eca2 P a72b3920c721481e90c181aa5bc059ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a72b3920c721481e90c181aa5bc059ee, State: Initialized, Role: FOLLOWER
I20260808 06:05:34.095937 18686 ts_tablet_manager.cc:1428] T 779fcf76782148ce9581af51fd87eca2 P d0f640df97f540d5878c0e44f839cba2: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260808 06:05:34.096172 18688 consensus_queue.cc:260] T 779fcf76782148ce9581af51fd87eca2 P a72b3920c721481e90c181aa5bc059ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17007b4f9a854b96a5f9bb31814441ff" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 41359 } } peers { permanent_uuid: "a72b3920c721481e90c181aa5bc059ee" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 46841 } } peers { permanent_uuid: "d0f640df97f540d5878c0e44f839cba2" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 36419 } }
I20260808 06:05:34.096772 18428 heartbeater.cc:503] Master 127.1.151.190:32893 was elected leader, sending a full tablet report...
I20260808 06:05:34.097103 18688 ts_tablet_manager.cc:1428] T 779fcf76782148ce9581af51fd87eca2 P a72b3920c721481e90c181aa5bc059ee: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 06:05:34.098732 18690 raft_consensus.cc:348] T 779fcf76782148ce9581af51fd87eca2 P 17007b4f9a854b96a5f9bb31814441ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17007b4f9a854b96a5f9bb31814441ff" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 41359 } } peers { permanent_uuid: "a72b3920c721481e90c181aa5bc059ee" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 46841 } } peers { permanent_uuid: "d0f640df97f540d5878c0e44f839cba2" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 36419 } }
I20260808 06:05:34.098922 18690 raft_consensus.cc:374] T 779fcf76782148ce9581af51fd87eca2 P 17007b4f9a854b96a5f9bb31814441ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:05:34.098971 18690 raft_consensus.cc:729] T 779fcf76782148ce9581af51fd87eca2 P 17007b4f9a854b96a5f9bb31814441ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17007b4f9a854b96a5f9bb31814441ff, State: Initialized, Role: FOLLOWER
I20260808 06:05:34.099357 18690 consensus_queue.cc:260] T 779fcf76782148ce9581af51fd87eca2 P 17007b4f9a854b96a5f9bb31814441ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17007b4f9a854b96a5f9bb31814441ff" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 41359 } } peers { permanent_uuid: "a72b3920c721481e90c181aa5bc059ee" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 46841 } } peers { permanent_uuid: "d0f640df97f540d5878c0e44f839cba2" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 36419 } }
I20260808 06:05:34.100261 18690 ts_tablet_manager.cc:1428] T 779fcf76782148ce9581af51fd87eca2 P 17007b4f9a854b96a5f9bb31814441ff: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 06:05:34.100678 18216 heartbeater.cc:503] Master 127.1.151.190:32893 was elected leader, sending a full tablet report...
I20260808 06:05:34.207937 18697 raft_consensus.cc:482] T 779fcf76782148ce9581af51fd87eca2 P 17007b4f9a854b96a5f9bb31814441ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:05:34.208214 18697 raft_consensus.cc:504] T 779fcf76782148ce9581af51fd87eca2 P 17007b4f9a854b96a5f9bb31814441ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17007b4f9a854b96a5f9bb31814441ff" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 41359 } } peers { permanent_uuid: "a72b3920c721481e90c181aa5bc059ee" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 46841 } } peers { permanent_uuid: "d0f640df97f540d5878c0e44f839cba2" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 36419 } }
I20260808 06:05:34.209376 18697 leader_election.cc:288] T 779fcf76782148ce9581af51fd87eca2 P 17007b4f9a854b96a5f9bb31814441ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a72b3920c721481e90c181aa5bc059ee (127.1.151.130:46841), d0f640df97f540d5878c0e44f839cba2 (127.1.151.131:36419)
I20260808 06:05:34.214756 18684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:34.209795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:51615 (local address 127.1.151.130:46841)
0808 06:05:34.209972 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:34.209976 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:34.210108 (+ 132us) server_negotiation.cc:407] Connection header received
0808 06:05:34.210310 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:34.210313 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:34.210366 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:34.210447 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:05:34.210832 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:34.211669 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:34.213377 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:34.214044 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:34.214188 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:34.214245 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:34.214526 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 06:05:34.214656 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260808 06:05:34.214740 18685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:34.209751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:52161 (local address 127.1.151.131:36419)
0808 06:05:34.209904 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:34.209909 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:34.210238 (+ 329us) server_negotiation.cc:407] Connection header received
0808 06:05:34.210309 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:34.210313 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:34.210364 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:34.210445 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:05:34.210832 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:34.211646 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:34.213377 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:34.214012 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:34.214188 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:34.214245 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:34.214526 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 06:05:34.214604 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 06:05:34.214789 18698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:34.209619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:46841 (local address 127.1.151.129:51615)
0808 06:05:34.209981 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:34.209998 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:34.210132 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:34.210472 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:34.210475 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:34.210508 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:05:34.210706 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:34.210714 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:34.211767 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:34.211771 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:34.213231 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:34.213243 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:34.214191 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:34.214195 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:34.214365 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:34.214424 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 06:05:34.214504 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 06:05:34.214611 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":201,"spinlock_wait_cycles":7808,"thread_start_us":97,"threads_started":1}
I20260808 06:05:34.214792 18700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:34.209664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:36419 (local address 127.1.151.129:52161)
0808 06:05:34.210107 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:34.210119 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 06:05:34.210232 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:05:34.210472 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:05:34.210475 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:05:34.210511 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:05:34.210711 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:34.210717 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:34.211767 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:34.211771 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:34.213231 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:05:34.213243 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:05:34.214191 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:05:34.214195 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:05:34.214362 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:34.214425 (+ 63us) client_negotiation.cc:769] Sending connection context
0808 06:05:34.214504 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 06:05:34.214608 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"mutex_wait_us":181,"spinlock_wait_cycles":6400,"thread_start_us":94,"threads_started":1}
I20260808 06:05:34.215562 18351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "779fcf76782148ce9581af51fd87eca2" candidate_uuid: "17007b4f9a854b96a5f9bb31814441ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a72b3920c721481e90c181aa5bc059ee" is_pre_election: true
I20260808 06:05:34.215644 18574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "779fcf76782148ce9581af51fd87eca2" candidate_uuid: "17007b4f9a854b96a5f9bb31814441ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f640df97f540d5878c0e44f839cba2" is_pre_election: true
I20260808 06:05:34.215876 18351 raft_consensus.cc:2454] T 779fcf76782148ce9581af51fd87eca2 P a72b3920c721481e90c181aa5bc059ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17007b4f9a854b96a5f9bb31814441ff in term 0.
I20260808 06:05:34.215905 18574 raft_consensus.cc:2454] T 779fcf76782148ce9581af51fd87eca2 P d0f640df97f540d5878c0e44f839cba2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17007b4f9a854b96a5f9bb31814441ff in term 0.
I20260808 06:05:34.216349 18033 leader_election.cc:302] T 779fcf76782148ce9581af51fd87eca2 P 17007b4f9a854b96a5f9bb31814441ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17007b4f9a854b96a5f9bb31814441ff, d0f640df97f540d5878c0e44f839cba2; no voters:
I20260808 06:05:34.216625 18697 raft_consensus.cc:2781] T 779fcf76782148ce9581af51fd87eca2 P 17007b4f9a854b96a5f9bb31814441ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 06:05:34.216717 18697 raft_consensus.cc:482] T 779fcf76782148ce9581af51fd87eca2 P 17007b4f9a854b96a5f9bb31814441ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 06:05:34.216766 18697 raft_consensus.cc:3037] T 779fcf76782148ce9581af51fd87eca2 P 17007b4f9a854b96a5f9bb31814441ff [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:05:34.217782 18697 raft_consensus.cc:504] T 779fcf76782148ce9581af51fd87eca2 P 17007b4f9a854b96a5f9bb31814441ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17007b4f9a854b96a5f9bb31814441ff" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 41359 } } peers { permanent_uuid: "a72b3920c721481e90c181aa5bc059ee" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 46841 } } peers { permanent_uuid: "d0f640df97f540d5878c0e44f839cba2" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 36419 } }
I20260808 06:05:34.218099 18697 leader_election.cc:288] T 779fcf76782148ce9581af51fd87eca2 P 17007b4f9a854b96a5f9bb31814441ff [CANDIDATE]: Term 1 election: Requested vote from peers a72b3920c721481e90c181aa5bc059ee (127.1.151.130:46841), d0f640df97f540d5878c0e44f839cba2 (127.1.151.131:36419)
I20260808 06:05:34.218495 18351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "779fcf76782148ce9581af51fd87eca2" candidate_uuid: "17007b4f9a854b96a5f9bb31814441ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a72b3920c721481e90c181aa5bc059ee"
I20260808 06:05:34.218495 18574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "779fcf76782148ce9581af51fd87eca2" candidate_uuid: "17007b4f9a854b96a5f9bb31814441ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f640df97f540d5878c0e44f839cba2"
I20260808 06:05:34.218621 18351 raft_consensus.cc:3037] T 779fcf76782148ce9581af51fd87eca2 P a72b3920c721481e90c181aa5bc059ee [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:05:34.218621 18574 raft_consensus.cc:3037] T 779fcf76782148ce9581af51fd87eca2 P d0f640df97f540d5878c0e44f839cba2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:05:34.219904 18351 raft_consensus.cc:2454] T 779fcf76782148ce9581af51fd87eca2 P a72b3920c721481e90c181aa5bc059ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17007b4f9a854b96a5f9bb31814441ff in term 1.
I20260808 06:05:34.219903 18574 raft_consensus.cc:2454] T 779fcf76782148ce9581af51fd87eca2 P d0f640df97f540d5878c0e44f839cba2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17007b4f9a854b96a5f9bb31814441ff in term 1.
I20260808 06:05:34.220153 18033 leader_election.cc:302] T 779fcf76782148ce9581af51fd87eca2 P 17007b4f9a854b96a5f9bb31814441ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17007b4f9a854b96a5f9bb31814441ff, d0f640df97f540d5878c0e44f839cba2; no voters:
I20260808 06:05:34.220302 18697 raft_consensus.cc:2781] T 779fcf76782148ce9581af51fd87eca2 P 17007b4f9a854b96a5f9bb31814441ff [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:05:34.220531 18697 raft_consensus.cc:686] T 779fcf76782148ce9581af51fd87eca2 P 17007b4f9a854b96a5f9bb31814441ff [term 1 LEADER]: Becoming Leader. State: Replica: 17007b4f9a854b96a5f9bb31814441ff, State: Running, Role: LEADER
I20260808 06:05:34.220836 18697 consensus_queue.cc:237] T 779fcf76782148ce9581af51fd87eca2 P 17007b4f9a854b96a5f9bb31814441ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17007b4f9a854b96a5f9bb31814441ff" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 41359 } } peers { permanent_uuid: "a72b3920c721481e90c181aa5bc059ee" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 46841 } } peers { permanent_uuid: "d0f640df97f540d5878c0e44f839cba2" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 36419 } }
I20260808 06:05:34.223976 17915 catalog_manager.cc:5495] T 779fcf76782148ce9581af51fd87eca2 P 17007b4f9a854b96a5f9bb31814441ff reported cstate change: term changed from 0 to 1, leader changed from <none> to 17007b4f9a854b96a5f9bb31814441ff (127.1.151.129). New cstate: current_term: 1 leader_uuid: "17007b4f9a854b96a5f9bb31814441ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17007b4f9a854b96a5f9bb31814441ff" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 41359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a72b3920c721481e90c181aa5bc059ee" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 46841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0f640df97f540d5878c0e44f839cba2" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 36419 } health_report { overall_health: UNKNOWN } } }
I20260808 06:05:34.264428 18679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:34.261037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47978 (local address 127.1.151.129:41359)
0808 06:05:34.261203 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 06:05:34.261207 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:05:34.261220 (+ 13us) server_negotiation.cc:407] Connection header received
0808 06:05:34.261301 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:05:34.261304 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:05:34.261363 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:05:34.261467 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:05:34.261683 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:34.262232 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:34.263051 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:05:34.263272 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:05:34.263369 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:05:34.263664 (+ 295us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:05:34.264106 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:05:34.264200 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 06:05:34.264259 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 06:05:34.264315 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 06:05:34.270220 18574 raft_consensus.cc:1261] T 779fcf76782148ce9581af51fd87eca2 P d0f640df97f540d5878c0e44f839cba2 [term 1 FOLLOWER]: Refusing update from remote peer 17007b4f9a854b96a5f9bb31814441ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:05:34.270222 18351 raft_consensus.cc:1261] T 779fcf76782148ce9581af51fd87eca2 P a72b3920c721481e90c181aa5bc059ee [term 1 FOLLOWER]: Refusing update from remote peer 17007b4f9a854b96a5f9bb31814441ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:05:34.270912 18701 consensus_queue.cc:1035] T 779fcf76782148ce9581af51fd87eca2 P 17007b4f9a854b96a5f9bb31814441ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a72b3920c721481e90c181aa5bc059ee" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 46841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:05:34.271081 18697 consensus_queue.cc:1035] T 779fcf76782148ce9581af51fd87eca2 P 17007b4f9a854b96a5f9bb31814441ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f640df97f540d5878c0e44f839cba2" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 36419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 06:05:34.277639 18667 tablet.cc:2369] T 779fcf76782148ce9581af51fd87eca2 P d0f640df97f540d5878c0e44f839cba2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 06:05:34.278045 18705 mvcc.cc:205] Tried to move back new op lower bound from 7316148773963923456 to 7316148773771218944. Current Snapshot: MvccSnapshot[applied={T|T < 7316148773963923456}]
I20260808 06:05:34.279052 18707 mvcc.cc:205] Tried to move back new op lower bound from 7316148773963923456 to 7316148773771218944. Current Snapshot: MvccSnapshot[applied={T|T < 7316148773963923456}]
I20260808 06:05:34.281257 18706 mvcc.cc:205] Tried to move back new op lower bound from 7316148773963923456 to 7316148773771218944. Current Snapshot: MvccSnapshot[applied={T|T < 7316148773963923456}]
I20260808 06:05:34.288539 17913 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42278:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:05:34.288707 17913 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42278:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:05:34.290983 17913 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 09912ce896fc46fb9474d451d2b52d4f: Sending DeleteTablet for 3 replicas of tablet 779fcf76782148ce9581af51fd87eca2
I20260808 06:05:34.291764 18327 tablet_service.cc:1514] Processing DeleteTablet for tablet 779fcf76782148ce9581af51fd87eca2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:05:34 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:43982
I20260808 06:05:34.291929 18104 tablet_service.cc:1514] Processing DeleteTablet for tablet 779fcf76782148ce9581af51fd87eca2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:05:34 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:47966
I20260808 06:05:34.292239 18722 tablet_replica.cc:330] T 779fcf76782148ce9581af51fd87eca2 P a72b3920c721481e90c181aa5bc059ee: stopping tablet replica
I20260808 06:05:34.292418 18725 tablet_replica.cc:330] T 779fcf76782148ce9581af51fd87eca2 P 17007b4f9a854b96a5f9bb31814441ff: stopping tablet replica
I20260808 06:05:34.292497 18545 tablet_service.cc:1514] Processing DeleteTablet for tablet 779fcf76782148ce9581af51fd87eca2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:05:34 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:47562
I20260808 06:05:34.292537 18722 raft_consensus.cc:2229] T 779fcf76782148ce9581af51fd87eca2 P a72b3920c721481e90c181aa5bc059ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:05:34.292809 18725 raft_consensus.cc:2229] T 779fcf76782148ce9581af51fd87eca2 P 17007b4f9a854b96a5f9bb31814441ff [term 1 LEADER]: Raft consensus shutting down.
I20260808 06:05:34.292841 18722 raft_consensus.cc:2258] T 779fcf76782148ce9581af51fd87eca2 P a72b3920c721481e90c181aa5bc059ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:05:34.293241 18725 raft_consensus.cc:2258] T 779fcf76782148ce9581af51fd87eca2 P 17007b4f9a854b96a5f9bb31814441ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:05:34.293262 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 18011
I20260808 06:05:34.294361 18722 ts_tablet_manager.cc:1905] T 779fcf76782148ce9581af51fd87eca2 P a72b3920c721481e90c181aa5bc059ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:05:34.297370 18722 ts_tablet_manager.cc:1918] T 779fcf76782148ce9581af51fd87eca2 P a72b3920c721481e90c181aa5bc059ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:05:34.297482 18722 log.cc:1196] T 779fcf76782148ce9581af51fd87eca2 P a72b3920c721481e90c181aa5bc059ee: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-1/wal/wals/779fcf76782148ce9581af51fd87eca2
I20260808 06:05:34.297854 18722 ts_tablet_manager.cc:1939] T 779fcf76782148ce9581af51fd87eca2 P a72b3920c721481e90c181aa5bc059ee: Deleting consensus metadata
I20260808 06:05:34.298964 18727 tablet_replica.cc:330] T 779fcf76782148ce9581af51fd87eca2 P d0f640df97f540d5878c0e44f839cba2: stopping tablet replica
I20260808 06:05:34.299261 18727 raft_consensus.cc:2229] T 779fcf76782148ce9581af51fd87eca2 P d0f640df97f540d5878c0e44f839cba2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:05:34.299247 17885 catalog_manager.cc:4841] TS a72b3920c721481e90c181aa5bc059ee (127.1.151.130:46841): tablet 779fcf76782148ce9581af51fd87eca2 (table test-table [id=2efb9f4989834581936d5fe8d5ab9b0a]) successfully deleted
I20260808 06:05:34.299546 18727 raft_consensus.cc:2258] T 779fcf76782148ce9581af51fd87eca2 P d0f640df97f540d5878c0e44f839cba2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:05:34.300695 18727 ts_tablet_manager.cc:1905] T 779fcf76782148ce9581af51fd87eca2 P d0f640df97f540d5878c0e44f839cba2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:05:34.303164 18727 ts_tablet_manager.cc:1918] T 779fcf76782148ce9581af51fd87eca2 P d0f640df97f540d5878c0e44f839cba2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:05:34.303268 18727 log.cc:1196] T 779fcf76782148ce9581af51fd87eca2 P d0f640df97f540d5878c0e44f839cba2: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786169070948234-1630-0/minicluster-data/ts-2/wal/wals/779fcf76782148ce9581af51fd87eca2
I20260808 06:05:34.303580 18727 ts_tablet_manager.cc:1939] T 779fcf76782148ce9581af51fd87eca2 P d0f640df97f540d5878c0e44f839cba2: Deleting consensus metadata
I20260808 06:05:34.304704 17882 catalog_manager.cc:4841] TS d0f640df97f540d5878c0e44f839cba2 (127.1.151.131:36419): tablet 779fcf76782148ce9581af51fd87eca2 (table test-table [id=2efb9f4989834581936d5fe8d5ab9b0a]) successfully deleted
W20260808 06:05:34.305408 17882 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.151.129:41359 (error 108)
I20260808 06:05:34.306241 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 18223
I20260808 06:05:34.306428 18677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:05:34.306128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:41359 (local address 127.0.0.1:47990)
0808 06:05:34.306277 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 06:05:34.306350 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.129:41359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260808 06:05:34.311421 17882 catalog_manager.cc:4584] TS 17007b4f9a854b96a5f9bb31814441ff (127.1.151.129:41359): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 779fcf76782148ce9581af51fd87eca2: Network error: Client connection negotiation failed: client connection to 127.1.151.129:41359: connect: Connection refused (error 111)
I20260808 06:05:34.321625 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 18436
I20260808 06:05:34.334297 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 17864
2026-08-08T06:05:34Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2478 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:05:34 dist-test-slave-zj8q krb5kdc[18736](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:05:34 dist-test-slave-zj8q krb5kdc[18736](info): set up 2 sockets
Aug 08 06:05:34 dist-test-slave-zj8q krb5kdc[18736](info): commencing operation
krb5kdc: starting...
W20260808 06:05:35.503126 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:05:35 dist-test-slave-zj8q krb5kdc[18736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:05:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:05:35Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.151.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.151.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.151.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/rangeradmin_127.1.151.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.151.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/rangeradmin_127.1.151.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.151.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.151.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.151.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/rangerlookup_127.1.151.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.151.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/rangerlookup_127.1.151.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.151.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.151.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.151.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/HTTP_127.1.151.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.151.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/HTTP_127.1.151.148@KRBTEST.COM.keytab.
I20260808 06:05:35.666409 1630 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskIVYy3F/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/postgres -l logfile start
2026-08-08 06:05:37.908 UTC [18775] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 06:05:37.908 UTC [18775] LOG: listening on IPv4 address "127.1.151.148", port 60749
2026-08-08 06:05:37.911 UTC [18775] LOG: listening on Unix socket "/tmp/.s.PGSQL.60749"
2026-08-08 06:05:37.931 UTC [18779] LOG: database system was shut down at 2026-08-08 06:05:36 UTC
2026-08-08 06:05:37.934 UTC [18775] LOG: database system is ready to accept connections
I20260808 06:05:38.997351 1630 mini_postgres.cc:96] Postgres bound to 60749
2026-08-08 06:05:39.004 UTC [18789] FATAL: database "slave" does not exist
127.1.151.148:60749 - accepting connections
I20260808 06:05:39.005196 1630 mini_ranger.cc:161] Starting Ranger...
I20260808 06:05:39.021945 1630 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 06:05:39.439427 1630 mini_ranger.cc:87] Created ranger Postgres database
I20260808 06:05:39.439527 1630 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ranger-admin
2026-08-08 06:05:39,746 [I] DB FLAVOR :POSTGRES
2026-08-08 06:05:39,746 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 06:05:39,746 [I] Checking connection..
2026-08-08 06:05:39,746 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:60749/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 06:05:39,981 [I] Checking connection passed.
2026-08-08 06:05:39,982 [I] --------- Verifying version history table ---------
2026-08-08 06:05:39,982 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:60749/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 06:05:40,223 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 06:05:40,224 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:60749/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 06:05:40,471 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 06:05:40,471 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 06:05:40,471 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:60749/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 06:05:40.697 UTC [18884] WARNING: there is no transaction in progress
2026-08-08 06:05:40,705 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 06:05:40,706 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:60749/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 06:05:40,961 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 06:05:40,962 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 06:05:40,962 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:60749/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 06:05:41,215 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:60749/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 06:05:41,472 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:60749/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-zj8q.c.gcp-upstream.internal','N') ;"
2026-08-08 06:05:41,708 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:60749/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 06:05:41,967 [I] Table x_portal_user does not exist in database ranger
2026-08-08 06:05:41,967 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:60749/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 06:05:42,228 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 06:05:42,228 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:60749/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 06:05:42,481 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:60749/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 06:05:42,724 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 06:05:42,725 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:60749/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 06:05:43.624 UTC [19047] WARNING: there is no transaction in progress
2026-08-08 06:05:43.639 UTC [19047] WARNING: there is no transaction in progress
2026-08-08 06:05:43.653 UTC [19047] WARNING: there is no transaction in progress
2026-08-08 06:05:43.667 UTC [19047] WARNING: there is no transaction in progress
2026-08-08 06:05:43.682 UTC [19047] WARNING: there is no transaction in progress
2026-08-08 06:05:43.699 UTC [19047] WARNING: there is no transaction in progress
2026-08-08 06:05:43.751 UTC [19047] WARNING: there is no transaction in progress
2026-08-08 06:05:43.757 UTC [19047] WARNING: there is no transaction in progress
2026-08-08 06:05:43.770 UTC [19047] WARNING: there is no transaction in progress
2026-08-08 06:05:43.779 UTC [19047] WARNING: there is no transaction in progress
2026-08-08 06:05:43.789 UTC [19047] WARNING: there is no transaction in progress
2026-08-08 06:05:43.802 UTC [19047] WARNING: there is no transaction in progress
2026-08-08 06:05:43.811 UTC [19047] WARNING: there is no transaction in progress
2026-08-08 06:05:43.816 UTC [19047] WARNING: there is no transaction in progress
2026-08-08 06:05:43.822 UTC [19047] WARNING: there is no transaction in progress
2026-08-08 06:05:43.827 UTC [19047] WARNING: there is no transaction in progress
2026-08-08 06:05:44,423 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 06:05:44,423 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:60749/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 06:05:44,669 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 06:05:44,669 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:60749/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 06:05:44,914 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 06:05:44,914 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:60749/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 06:05:45,175 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 06:05:45,175 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:60749/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 06:05:45,421 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:60749/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 06:05:45,668 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:60749/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-zj8q.c.gcp-upstream.internal';"
2026-08-08 06:05:45,917 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 06:05:45,918 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:60749/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 06:05:46,177 [I] DB_PATCHES have already been applied
I20260808 06:05:46.183503 1630 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ranger-admin:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskIVYy3F/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 06:05:46.183581 1630 mini_ranger.cc:193] Using host: 127.1.151.148
I20260808 06:05:46.188158 1630 mini_ranger.cc:227] Ranger admin URL: http://127.1.151.148:48721
Aug 08, 2026 6:05:46 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 6:05:46 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 6:05:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 6:05:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 6:05:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 6:05:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 6:05:47 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 06:05:47.447000 1630 mini_ranger.cc:160] Time spent starting Ranger: real 8.442s user 0.000s sys 0.013s
Aug 08, 2026 6:06:07 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 6:06:07 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 6:06:07 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 6:06:07 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 6:06:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 6:06:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 6:06:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 6:06:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 6:06:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 6:06:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 6:06:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 6:06:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 6:06:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 6:06:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 6:06:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 6:06:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 6:06:07 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 6:06:08 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 06:06:09.043540 1630 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.151.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.151.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.151.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/rangerkms_127.1.151.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.151.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/rangerkms_127.1.151.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.151.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.151.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.151.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/HTTP_127.1.151.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.151.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/HTTP_127.1.151.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 06:06:09 dist-test-slave-zj8q krb5kdc[18736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169169, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 06:06:09.137586 1630 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 06:06:09.147580 1630 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 06:06:10.003677 1630 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 06:06:10.003783 1630 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ranger-kms
I20260808 06:06:10.003883 1630 mini_ranger_kms.cc:227] Using postgres at 127.1.151.148:60749
2026-08-08 06:06:10,638 [I] DB FLAVOR :POSTGRES
2026-08-08 06:06:10,638 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 06:06:10,639 [I] Checking connection
2026-08-08 06:06:10,639 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:60749/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 06:06:10,931 [I] connection success
2026-08-08 06:06:10,931 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 06:06:10,931 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 06:06:10,931 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:60749/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 06:06:11,196 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 06:06:11,196 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 06:06:11,196 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 06:06:11,196 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:60749/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 06:06:11,447 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 06:06:11.920647 1630 mini_ranger_kms.cc:310] Created kms service
I20260808 06:06:12.111065 1630 mini_ranger_kms.cc:326] Created kudu user
I20260808 06:06:12.159811 1630 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 06:06:12 dist-test-slave-zj8q krb5kdc[18736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169169, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.151.148@KRBTEST.COM
I20260808 06:06:12.540298 1630 mini_ranger_kms.cc:384] Added ranger policy
I20260808 06:06:12.540457 1630 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ranger-kms:/tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 06:06:12.540479 1630 mini_ranger_kms.cc:240] Using host: 127.1.151.148
I20260808 06:06:12.545492 1630 mini_ranger_kms.cc:276] Ranger KMS PID: 19322
I20260808 06:06:12.545676 1630 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.151.148:54839
Aug 08, 2026 6:06:13 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 6:06:13 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 6:06:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 6:06:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 6:06:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 6:06:14 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260808 06:06:15.156317 1630 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.019s user 0.012s sys 0.591s
I20260808 06:06:15.156468 1630 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 06:06:15.156612 1630 mini_ranger_kms.cc:401] 127.1.151.148:54839/kms/v1/keys
Aug 08 06:06:15 dist-test-slave-zj8q krb5kdc[18736](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786169175, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.151.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 08 06:06:15 dist-test-slave-zj8q krb5kdc[18736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169175, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.151.148@KRBTEST.COM for HTTP/127.1.151.148@KRBTEST.COM
Aug 08 06:06:16 dist-test-slave-zj8q krb5kdc[18736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169175, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.151.148@KRBTEST.COM for HTTP/127.1.151.148@KRBTEST.COM
Aug 08 06:06:18 dist-test-slave-zj8q krb5kdc[18736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.151.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
I20260808 06:06:18.489292 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:45561
--webserver_interface=127.1.151.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:35429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:45561
--ranger_config_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.151.148:54839/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:06:18.655413 19394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:06:18.655659 19394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:06:18.655699 19394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:06:18.659646 19394 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:06:18.659698 19394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:06:18.659715 19394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:06:18.659730 19394 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:06:18.659744 19394 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:06:18.665515 19394 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:35429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.151.148:54839/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:45561
--ranger_config_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:45561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:06:18.666561 19394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:06:18.667572 19394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:06:18.677625 19403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:06:18.677644 19409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:06:18.677721 19407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:06:18.678337 19394 server_base.cc:1034] running on GCE node
I20260808 06:06:18.679061 19394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:06:18.680507 19394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:06:18.681694 19394 hybrid_clock.cc:648] HybridClock initialized: now 1786169178681667 us; error 33 us; skew 500 ppm
Aug 08 06:06:18 dist-test-slave-zj8q krb5kdc[18736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169178, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:06:18.685204 19394 init.cc:377] Logged in from keytab as kudu/127.1.151.190@KRBTEST.COM (short username kudu)
I20260808 06:06:18.686358 19394 webserver.cc:460] Webserver started at http://127.1.151.190:41801/ using document root <none> and password file <none>
I20260808 06:06:18.687083 19394 fs_manager.cc:359] Metadata directory not provided
I20260808 06:06:18.687191 19394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:06:18.687443 19394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 06:06:18 dist-test-slave-zj8q krb5kdc[18736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169178, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for HTTP/127.1.151.148@KRBTEST.COM
I20260808 06:06:18.740228 19394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "ae702375cd0c41f5be2b5b4baa7f37a6"
format_stamp: "Formatted at 2026-08-08 06:06:18 on dist-test-slave-zj8q"
server_key: "313f62da5fd02b95a9559e89d6d9ba42"
server_key_iv: "73d8bd6118964f502fc88517ab59a178"
server_key_version: "kuduclusterkey@0"
I20260808 06:06:18.740903 19394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "ae702375cd0c41f5be2b5b4baa7f37a6"
format_stamp: "Formatted at 2026-08-08 06:06:18 on dist-test-slave-zj8q"
server_key: "313f62da5fd02b95a9559e89d6d9ba42"
server_key_iv: "73d8bd6118964f502fc88517ab59a178"
server_key_version: "kuduclusterkey@0"
I20260808 06:06:18.745157 19394 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260808 06:06:18.758929 19419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:06:18.760334 19394 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 06:06:18.760480 19394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "ae702375cd0c41f5be2b5b4baa7f37a6"
format_stamp: "Formatted at 2026-08-08 06:06:18 on dist-test-slave-zj8q"
server_key: "313f62da5fd02b95a9559e89d6d9ba42"
server_key_iv: "73d8bd6118964f502fc88517ab59a178"
server_key_version: "kuduclusterkey@0"
I20260808 06:06:18.760617 19394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:06:18.773937 19394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:06:18.776958 19394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:06:18.777138 19394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:06:18.794026 19394 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:45561
I20260808 06:06:18.794225 19491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:45561 every 8 connection(s)
I20260808 06:06:18.795195 19394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:06:18.797070 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 19394
I20260808 06:06:18.797196 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:06:18.797560 1630 external_mini_cluster.cc:1428] Setting key 1b1548f075fa01bf837fb4a3fcf39068
I20260808 06:06:18.798573 19492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:06:18.807402 19492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae702375cd0c41f5be2b5b4baa7f37a6: Bootstrap starting.
Aug 08 06:06:18 dist-test-slave-zj8q krb5kdc[18736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:06:18.810854 19492 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae702375cd0c41f5be2b5b4baa7f37a6: Neither blocks nor log segments found. Creating new log.
I20260808 06:06:18.812479 19492 log.cc:824] T 00000000000000000000000000000000 P ae702375cd0c41f5be2b5b4baa7f37a6: Log is configured to *not* fsync() on all Append() calls
I20260808 06:06:18.814941 19492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae702375cd0c41f5be2b5b4baa7f37a6: No bootstrap required, opened a new log
I20260808 06:06:18.818943 19496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:06:18.799047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60542 (local address 127.1.151.190:45561)
0808 06:06:18.799920 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0808 06:06:18.799991 (+ 71us) server_negotiation.cc:399] Waiting for connection header
0808 06:06:18.800042 (+ 51us) server_negotiation.cc:407] Connection header received
0808 06:06:18.801611 (+ 1569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:06:18.801638 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:06:18.802099 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:06:18.802514 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:06:18.803573 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:06:18.805610 (+ 2037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:06:18.806825 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:06:18.807647 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:06:18.807810 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:06:18.811308 (+ 3498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:06:18.811318 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:06:18.811338 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:06:18.811373 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:06:18.815308 (+ 3935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:06:18.815824 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:06:18.815830 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:06:18.815836 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:06:18.815985 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:06:18.816310 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:06:18.816313 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:06:18.816315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:06:18.816832 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:06:18.817050 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0808 06:06:18.817549 (+ 499us) server_negotiation.cc:299] Negotiation successful
0808 06:06:18.817833 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":698,"thread_start_us":217,"threads_started":1}
I20260808 06:06:18.820294 19492 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae702375cd0c41f5be2b5b4baa7f37a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae702375cd0c41f5be2b5b4baa7f37a6" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 45561 } }
I20260808 06:06:18.820515 19492 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae702375cd0c41f5be2b5b4baa7f37a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:06:18.820600 19492 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae702375cd0c41f5be2b5b4baa7f37a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae702375cd0c41f5be2b5b4baa7f37a6, State: Initialized, Role: FOLLOWER
I20260808 06:06:18.821055 19492 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae702375cd0c41f5be2b5b4baa7f37a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae702375cd0c41f5be2b5b4baa7f37a6" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 45561 } }
I20260808 06:06:18.821194 19492 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae702375cd0c41f5be2b5b4baa7f37a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:06:18.821276 19492 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae702375cd0c41f5be2b5b4baa7f37a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:06:18.821376 19492 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae702375cd0c41f5be2b5b4baa7f37a6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:06:18.822659 19492 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae702375cd0c41f5be2b5b4baa7f37a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae702375cd0c41f5be2b5b4baa7f37a6" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 45561 } }
I20260808 06:06:18.822995 19492 leader_election.cc:302] T 00000000000000000000000000000000 P ae702375cd0c41f5be2b5b4baa7f37a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae702375cd0c41f5be2b5b4baa7f37a6; no voters:
I20260808 06:06:18.823324 19492 leader_election.cc:288] T 00000000000000000000000000000000 P ae702375cd0c41f5be2b5b4baa7f37a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:06:18.823524 19499 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae702375cd0c41f5be2b5b4baa7f37a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:06:18.823879 19499 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae702375cd0c41f5be2b5b4baa7f37a6 [term 1 LEADER]: Becoming Leader. State: Replica: ae702375cd0c41f5be2b5b4baa7f37a6, State: Running, Role: LEADER
I20260808 06:06:18.824214 19499 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae702375cd0c41f5be2b5b4baa7f37a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae702375cd0c41f5be2b5b4baa7f37a6" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 45561 } }
I20260808 06:06:18.824653 19492 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae702375cd0c41f5be2b5b4baa7f37a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:06:18.826988 19504 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae702375cd0c41f5be2b5b4baa7f37a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae702375cd0c41f5be2b5b4baa7f37a6. Latest consensus state: current_term: 1 leader_uuid: "ae702375cd0c41f5be2b5b4baa7f37a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae702375cd0c41f5be2b5b4baa7f37a6" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 45561 } } }
I20260808 06:06:18.827359 19500 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae702375cd0c41f5be2b5b4baa7f37a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae702375cd0c41f5be2b5b4baa7f37a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae702375cd0c41f5be2b5b4baa7f37a6" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 45561 } } }
I20260808 06:06:18.827494 19500 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae702375cd0c41f5be2b5b4baa7f37a6 [sys.catalog]: This master's current role is: LEADER
I20260808 06:06:18.828383 19514 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:06:18.828882 19504 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae702375cd0c41f5be2b5b4baa7f37a6 [sys.catalog]: This master's current role is: LEADER
I20260808 06:06:18.832142 19514 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:06:18.832655 19492 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 06:06:18.843686 19514 catalog_manager.cc:1347] Generated new cluster ID: 5a86ef3b8e364d67bb3b80d4f6910d6d
I20260808 06:06:18.844105 19514 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:06:18.867753 19514 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:06:18.869130 19514 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:06:18.879479 19514 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae702375cd0c41f5be2b5b4baa7f37a6: Generated new TSK 0
I20260808 06:06:18.880378 19514 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.151.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab.
Entry for principal kudu/127.1.151.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab.
I20260808 06:06:19.863386 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.151.148:54839/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.129:0
--local_ip_for_outbound_sockets=127.1.151.129
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:45561
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:35429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:06:20.052690 19562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:06:20.052949 19562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:06:20.053011 19562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:06:20.057611 19562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:06:20.057675 19562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:06:20.057753 19562 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.129
I20260808 06:06:20.063377 19562 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:35429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.151.148:54839/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:45561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.129
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:06:20.064450 19562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:06:20.065539 19562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:06:20.077085 19569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:06:20.077318 19568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:06:20.077093 19572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:06:20.078471 19562 server_base.cc:1034] running on GCE node
I20260808 06:06:20.078929 19562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:06:20.079685 19562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:06:20.080874 19562 hybrid_clock.cc:648] HybridClock initialized: now 1786169180080844 us; error 44 us; skew 500 ppm
Aug 08 06:06:20 dist-test-slave-zj8q krb5kdc[18736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169180, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:06:20.084967 19562 init.cc:377] Logged in from keytab as kudu/127.1.151.129@KRBTEST.COM (short username kudu)
I20260808 06:06:20.086069 19562 webserver.cc:460] Webserver started at http://127.1.151.129:42463/ using document root <none> and password file <none>
I20260808 06:06:20.086833 19562 fs_manager.cc:359] Metadata directory not provided
I20260808 06:06:20.086937 19562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:06:20.087273 19562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 06:06:20 dist-test-slave-zj8q krb5kdc[18736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169180, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for HTTP/127.1.151.148@KRBTEST.COM
I20260808 06:06:20.105937 19562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-0/data/instance:
uuid: "313b40a3a8d84c63ba2dab93cc69b540"
format_stamp: "Formatted at 2026-08-08 06:06:20 on dist-test-slave-zj8q"
server_key: "396a6114cac2890387292be492e22c09"
server_key_iv: "56586a1d1344c0c24a5b4b0c8a73c79d"
server_key_version: "kuduclusterkey@0"
I20260808 06:06:20.106885 19562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance:
uuid: "313b40a3a8d84c63ba2dab93cc69b540"
format_stamp: "Formatted at 2026-08-08 06:06:20 on dist-test-slave-zj8q"
server_key: "396a6114cac2890387292be492e22c09"
server_key_iv: "56586a1d1344c0c24a5b4b0c8a73c79d"
server_key_version: "kuduclusterkey@0"
I20260808 06:06:20.112658 19562 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 06:06:20.132009 19584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:06:20.133517 19562 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 06:06:20.133652 19562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-0/wal
uuid: "313b40a3a8d84c63ba2dab93cc69b540"
format_stamp: "Formatted at 2026-08-08 06:06:20 on dist-test-slave-zj8q"
server_key: "396a6114cac2890387292be492e22c09"
server_key_iv: "56586a1d1344c0c24a5b4b0c8a73c79d"
server_key_version: "kuduclusterkey@0"
I20260808 06:06:20.133777 19562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:06:20.160579 19562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:06:20.164615 19562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:06:20.164938 19562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:06:20.165894 19562 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:06:20.167109 19562 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:06:20.167219 19562 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:06:20.167347 19562 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:06:20.167426 19562 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:06:20.188030 19562 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.129:42749
I20260808 06:06:20.188382 19753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.129:42749 every 8 connection(s)
I20260808 06:06:20.189002 19562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
I20260808 06:06:20.193754 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 19562
I20260808 06:06:20.193878 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance
I20260808 06:06:20.194303 1630 external_mini_cluster.cc:1428] Setting key 13404b3ee0e8a329ad0301ceb8c80623
Aug 08 06:06:20 dist-test-slave-zj8q krb5kdc[18736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169180, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:06:20.209007 19757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:06:20.191292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:56747 (local address 127.1.151.190:45561)
0808 06:06:20.191678 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0808 06:06:20.191683 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:06:20.193505 (+ 1822us) server_negotiation.cc:407] Connection header received
0808 06:06:20.195598 (+ 2093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:06:20.195602 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:06:20.195669 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:06:20.195736 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:06:20.197970 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:06:20.198822 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:06:20.200166 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:06:20.200512 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:06:20.200619 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:06:20.204157 (+ 3538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:06:20.204161 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:06:20.204163 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:06:20.204192 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:06:20.206080 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:06:20.206798 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:06:20.206803 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:06:20.206805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:06:20.206905 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:06:20.207376 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:06:20.207379 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:06:20.207381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:06:20.207623 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:06:20.207700 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 06:06:20.208669 (+ 969us) server_negotiation.cc:299] Negotiation successful
0808 06:06:20.208817 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":139,"threads_started":1}
I20260808 06:06:20.210176 19758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:06:20.191768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:45561 (local address 127.1.151.129:56747)
0808 06:06:20.193346 (+ 1578us) negotiation.cc:107] Waiting for socket to connect
0808 06:06:20.193392 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 06:06:20.195340 (+ 1948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:06:20.196270 (+ 930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:06:20.196281 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:06:20.196939 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:06:20.197790 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:06:20.197813 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:06:20.198980 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:06:20.198983 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:06:20.200032 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:06:20.200043 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:06:20.200808 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:06:20.200812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:06:20.201074 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:06:20.201965 (+ 891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:06:20.201989 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:06:20.203924 (+ 1935us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:06:20.206283 (+ 2359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:06:20.206290 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:06:20.206307 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:06:20.206667 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:06:20.207030 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:06:20.207033 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:06:20.207035 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:06:20.207250 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:06:20.207792 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:06:20.207799 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:06:20.208287 (+ 488us) client_negotiation.cc:769] Sending connection context
0808 06:06:20.208583 (+ 296us) client_negotiation.cc:240] Negotiation successful
0808 06:06:20.208949 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1350,"thread_start_us":276,"threads_started":1}
WARNING: no policy specified for kudu/127.1.151.130@KRBTEST.COM; defaulting to no policy
I20260808 06:06:20.212044 19754 heartbeater.cc:347] Connected to a master server at 127.1.151.190:45561
I20260808 06:06:20.212313 19754 heartbeater.cc:464] Registering TS with master...
I20260808 06:06:20.212935 19754 heartbeater.cc:511] Master 127.1.151.190:45561 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.130@KRBTEST.COM" created.
I20260808 06:06:20.214967 19439 ts_manager.cc:194] Registered new tserver with Master: 313b40a3a8d84c63ba2dab93cc69b540 (127.1.151.129:42749)
I20260808 06:06:20.216616 19439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.129@KRBTEST.COM'} at 127.1.151.129:56747
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab.
Entry for principal kudu/127.1.151.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab.
I20260808 06:06:20.235373 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.151.148:54839/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.130:0
--local_ip_for_outbound_sockets=127.1.151.130
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:45561
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:35429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 06:06:20 dist-test-slave-zj8q krb5kdc[18736](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786169180, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 06:06:20.449102 19767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:06:20.449540 19767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:06:20.449676 19767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:06:20.455710 19767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:06:20.455826 19767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:06:20.455945 19767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.130
I20260808 06:06:20.463778 19767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:35429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.151.148:54839/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:45561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.130
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:06:20.465314 19767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:06:20.466811 19767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:06:20.490720 19780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:06:20.491319 19779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:06:20.492502 19784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:06:20.494071 19767 server_base.cc:1034] running on GCE node
I20260808 06:06:20.494656 19767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:06:20.495539 19767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:06:20.496750 19767 hybrid_clock.cc:648] HybridClock initialized: now 1786169180496724 us; error 41 us; skew 500 ppm
Aug 08 06:06:20 dist-test-slave-zj8q krb5kdc[18736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169180, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:06:20.499732 19767 init.cc:377] Logged in from keytab as kudu/127.1.151.130@KRBTEST.COM (short username kudu)
I20260808 06:06:20.500602 19767 webserver.cc:460] Webserver started at http://127.1.151.130:36913/ using document root <none> and password file <none>
I20260808 06:06:20.501149 19767 fs_manager.cc:359] Metadata directory not provided
I20260808 06:06:20.501207 19767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:06:20.501386 19767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 06:06:20 dist-test-slave-zj8q krb5kdc[18736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169180, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for HTTP/127.1.151.148@KRBTEST.COM
I20260808 06:06:20.517431 19767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-1/data/instance:
uuid: "bb9051f3cd1543738bbbda9fa1d0b8e7"
format_stamp: "Formatted at 2026-08-08 06:06:20 on dist-test-slave-zj8q"
server_key: "36bd6a539409c167c7801f496865eaca"
server_key_iv: "32f27db3b57c6913688219ee9bb1167e"
server_key_version: "kuduclusterkey@0"
I20260808 06:06:20.518409 19767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance:
uuid: "bb9051f3cd1543738bbbda9fa1d0b8e7"
format_stamp: "Formatted at 2026-08-08 06:06:20 on dist-test-slave-zj8q"
server_key: "36bd6a539409c167c7801f496865eaca"
server_key_iv: "32f27db3b57c6913688219ee9bb1167e"
server_key_version: "kuduclusterkey@0"
I20260808 06:06:20.523582 19767 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 06:06:20.545945 19796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:06:20.547847 19767 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.001s sys 0.000s
I20260808 06:06:20.547991 19767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-1/wal
uuid: "bb9051f3cd1543738bbbda9fa1d0b8e7"
format_stamp: "Formatted at 2026-08-08 06:06:20 on dist-test-slave-zj8q"
server_key: "36bd6a539409c167c7801f496865eaca"
server_key_iv: "32f27db3b57c6913688219ee9bb1167e"
server_key_version: "kuduclusterkey@0"
I20260808 06:06:20.548149 19767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:06:20.577894 19767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:06:20.581655 19767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:06:20.581928 19767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:06:20.582737 19767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:06:20.583877 19767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:06:20.583983 19767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:06:20.584087 19767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:06:20.584180 19767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:06:20.609061 19767 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.130:44359
I20260808 06:06:20.610847 19767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
I20260808 06:06:20.619051 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 19767
I20260808 06:06:20.619182 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance
I20260808 06:06:20.619503 1630 external_mini_cluster.cc:1428] Setting key 1c974079be23eb4dedaa3563424fc0e0
I20260808 06:06:20.624099 19972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.130:44359 every 8 connection(s)
WARNING: no policy specified for kudu/127.1.151.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.131@KRBTEST.COM" created.
Aug 08 06:06:20 dist-test-slave-zj8q krb5kdc[18736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169180, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:06:20.657306 19757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:06:20.628222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:41277 (local address 127.1.151.190:45561)
0808 06:06:20.628389 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 06:06:20.628394 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:06:20.637715 (+ 9321us) server_negotiation.cc:407] Connection header received
0808 06:06:20.639057 (+ 1342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:06:20.639061 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:06:20.639122 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:06:20.639197 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:06:20.642266 (+ 3069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:06:20.643098 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:06:20.645938 (+ 2840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:06:20.646347 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:06:20.646448 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:06:20.653148 (+ 6700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:06:20.653152 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:06:20.653155 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:06:20.653187 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:06:20.654934 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:06:20.655528 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:06:20.655532 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:06:20.655534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:06:20.655631 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:06:20.656017 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:06:20.656021 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:06:20.656022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:06:20.656261 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:06:20.656365 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 06:06:20.656977 (+ 612us) server_negotiation.cc:299] Negotiation successful
0808 06:06:20.657129 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 06:06:20.658299 19976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:06:20.626901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:45561 (local address 127.1.151.130:41277)
0808 06:06:20.637507 (+ 10606us) negotiation.cc:107] Waiting for socket to connect
0808 06:06:20.637594 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0808 06:06:20.638790 (+ 1196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:06:20.640843 (+ 2053us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:06:20.640854 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:06:20.641250 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:06:20.642059 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:06:20.642083 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:06:20.644016 (+ 1933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:06:20.644020 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:06:20.645807 (+ 1787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:06:20.645820 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:06:20.650295 (+ 4475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:06:20.650300 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:06:20.650507 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:06:20.651243 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:06:20.651267 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:06:20.652970 (+ 1703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:06:20.655048 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:06:20.655054 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:06:20.655072 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:06:20.655410 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:06:20.655721 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:06:20.655725 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:06:20.655727 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:06:20.655900 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:06:20.656364 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:06:20.656371 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:06:20.656698 (+ 327us) client_negotiation.cc:769] Sending connection context
0808 06:06:20.656898 (+ 200us) client_negotiation.cc:240] Negotiation successful
0808 06:06:20.657232 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10316,"thread_start_us":192,"threads_started":1}
I20260808 06:06:20.659770 19973 heartbeater.cc:347] Connected to a master server at 127.1.151.190:45561
I20260808 06:06:20.660033 19973 heartbeater.cc:464] Registering TS with master...
I20260808 06:06:20.660666 19973 heartbeater.cc:511] Master 127.1.151.190:45561 requested a full tablet report, sending...
I20260808 06:06:20.661954 19439 ts_manager.cc:194] Registered new tserver with Master: bb9051f3cd1543738bbbda9fa1d0b8e7 (127.1.151.130:44359)
I20260808 06:06:20.662731 19439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.130@KRBTEST.COM'} at 127.1.151.130:41277
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab.
Entry for principal kudu/127.1.151.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab.
I20260808 06:06:20.679690 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.151.148:54839/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.131:0
--local_ip_for_outbound_sockets=127.1.151.131
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:45561
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:35429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:06:20.906322 19979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:06:20.906728 19979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:06:20.906867 19979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:06:20.912827 19979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:06:20.912896 19979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:06:20.912983 19979 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.131
I20260808 06:06:20.920849 19979 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:35429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.151.148:54839/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:45561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.131
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:06:20.922219 19979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:06:20.923599 19979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:06:20.937656 19987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:06:20.938067 19989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:06:20.938540 19985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:06:20.943020 19979 server_base.cc:1034] running on GCE node
I20260808 06:06:20.943658 19979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:06:20.944504 19979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:06:20.951943 19979 hybrid_clock.cc:648] HybridClock initialized: now 1786169180951867 us; error 108 us; skew 500 ppm
Aug 08 06:06:20 dist-test-slave-zj8q krb5kdc[18736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169180, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:06:20.956198 19979 init.cc:377] Logged in from keytab as kudu/127.1.151.131@KRBTEST.COM (short username kudu)
I20260808 06:06:20.957484 19979 webserver.cc:460] Webserver started at http://127.1.151.131:33059/ using document root <none> and password file <none>
I20260808 06:06:20.958324 19979 fs_manager.cc:359] Metadata directory not provided
I20260808 06:06:20.958427 19979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:06:20.958740 19979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 06:06:20 dist-test-slave-zj8q krb5kdc[18736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169180, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for HTTP/127.1.151.148@KRBTEST.COM
I20260808 06:06:20.981982 19979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-2/data/instance:
uuid: "917d9c3aff9942f483b4befb516f1ab0"
format_stamp: "Formatted at 2026-08-08 06:06:20 on dist-test-slave-zj8q"
server_key: "39bf499026f02cc8f74a4276e4677cfe"
server_key_iv: "dacec2f5b856d25b7bb1cae1e378a62d"
server_key_version: "kuduclusterkey@0"
I20260808 06:06:20.983174 19979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance:
uuid: "917d9c3aff9942f483b4befb516f1ab0"
format_stamp: "Formatted at 2026-08-08 06:06:20 on dist-test-slave-zj8q"
server_key: "39bf499026f02cc8f74a4276e4677cfe"
server_key_iv: "dacec2f5b856d25b7bb1cae1e378a62d"
server_key_version: "kuduclusterkey@0"
I20260808 06:06:20.988736 19979 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 06:06:21.015885 19997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:06:21.017616 19979 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 06:06:21.017761 19979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-2/wal
uuid: "917d9c3aff9942f483b4befb516f1ab0"
format_stamp: "Formatted at 2026-08-08 06:06:20 on dist-test-slave-zj8q"
server_key: "39bf499026f02cc8f74a4276e4677cfe"
server_key_iv: "dacec2f5b856d25b7bb1cae1e378a62d"
server_key_version: "kuduclusterkey@0"
I20260808 06:06:21.017892 19979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:06:21.040340 19979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
Aug 08 06:06:21 dist-test-slave-zj8q krb5kdc[18736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169180, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for HTTP/127.1.151.148@KRBTEST.COM
I20260808 06:06:21.044914 19979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:06:21.045207 19979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:06:21.046068 19979 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:06:21.047385 19979 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:06:21.047448 19979 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:06:21.047501 19979 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:06:21.047569 19979 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:06:21.072836 19979 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.131:41971
I20260808 06:06:21.074097 19979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
I20260808 06:06:21.080471 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 19979
I20260808 06:06:21.080600 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance
I20260808 06:06:21.080938 1630 external_mini_cluster.cc:1428] Setting key 139563ba0cda06e2dd60685cce4d56d4
I20260808 06:06:21.095738 20168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.131:41971 every 8 connection(s)
Aug 08 06:06:21 dist-test-slave-zj8q krb5kdc[18736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169180, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:06:21.119731 19757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:06:21.098314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:47189 (local address 127.1.151.190:45561)
0808 06:06:21.098432 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 06:06:21.098436 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:06:21.102813 (+ 4377us) server_negotiation.cc:407] Connection header received
0808 06:06:21.104287 (+ 1474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:06:21.104291 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:06:21.104346 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:06:21.104396 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:06:21.107589 (+ 3193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:06:21.108432 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:06:21.110654 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:06:21.111002 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:06:21.111095 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:06:21.114508 (+ 3413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:06:21.114513 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:06:21.114516 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:06:21.114545 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:06:21.116205 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:06:21.116796 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:06:21.116800 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:06:21.116802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:06:21.116886 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:06:21.117269 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:06:21.117273 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:06:21.117274 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:06:21.117483 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:06:21.117560 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 06:06:21.119433 (+ 1873us) server_negotiation.cc:299] Negotiation successful
0808 06:06:21.119565 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 06:06:21.121601 20172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:06:21.098097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:45561 (local address 127.1.151.131:47189)
0808 06:06:21.102576 (+ 4479us) negotiation.cc:107] Waiting for socket to connect
0808 06:06:21.102637 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 06:06:21.104028 (+ 1391us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:06:21.104828 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:06:21.104837 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:06:21.105352 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:06:21.107398 (+ 2046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:06:21.107422 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:06:21.108773 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:06:21.108776 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:06:21.110502 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:06:21.110515 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:06:21.111298 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:06:21.111302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:06:21.111613 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:06:21.112344 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:06:21.112365 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:06:21.114314 (+ 1949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:06:21.116333 (+ 2019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:06:21.116339 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:06:21.116355 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:06:21.116684 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:06:21.116983 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:06:21.116986 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:06:21.116988 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:06:21.117165 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:06:21.118128 (+ 963us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:06:21.118135 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:06:21.119149 (+ 1014us) client_negotiation.cc:769] Sending connection context
0808 06:06:21.120185 (+ 1036us) client_negotiation.cc:240] Negotiation successful
0808 06:06:21.120516 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4177,"thread_start_us":1482,"threads_started":1}
I20260808 06:06:21.123656 20170 heartbeater.cc:347] Connected to a master server at 127.1.151.190:45561
I20260808 06:06:21.123898 20170 heartbeater.cc:464] Registering TS with master...
I20260808 06:06:21.124445 20170 heartbeater.cc:511] Master 127.1.151.190:45561 requested a full tablet report, sending...
I20260808 06:06:21.125712 19439 ts_manager.cc:194] Registered new tserver with Master: 917d9c3aff9942f483b4befb516f1ab0 (127.1.151.131:41971)
I20260808 06:06:21.126433 19439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.131@KRBTEST.COM'} at 127.1.151.131:47189
I20260808 06:06:21.130985 1630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 06:06:21.143224 19757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:06:21.133653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60550 (local address 127.1.151.190:45561)
0808 06:06:21.133759 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 06:06:21.133763 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:06:21.134262 (+ 499us) server_negotiation.cc:407] Connection header received
0808 06:06:21.134451 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:06:21.134454 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:06:21.134498 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:06:21.134544 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:06:21.135533 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:06:21.136340 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:06:21.137568 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:06:21.137904 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:06:21.137971 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:06:21.139756 (+ 1785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:06:21.139760 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:06:21.139763 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:06:21.139788 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:06:21.141407 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:06:21.141886 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:06:21.141889 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:06:21.141891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:06:21.141970 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:06:21.142299 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:06:21.142303 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:06:21.142304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:06:21.142502 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:06:21.142599 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 06:06:21.142948 (+ 349us) server_negotiation.cc:299] Negotiation successful
0808 06:06:21.143072 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 06:06:21.146836 19439 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60550:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 06:06:21.151142 19439 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 06:06:21.177307 20184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:06:21.164007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:42749 (local address 127.0.0.1:42920)
0808 06:06:21.167885 (+ 3878us) negotiation.cc:107] Waiting for socket to connect
0808 06:06:21.167939 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 06:06:21.169783 (+ 1844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:06:21.170559 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:06:21.170565 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:06:21.170588 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:06:21.170820 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:06:21.170828 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:06:21.173105 (+ 2277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:06:21.173109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:06:21.174620 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:06:21.174630 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:06:21.176670 (+ 2040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:06:21.176674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:06:21.176863 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:06:21.176909 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 06:06:21.177018 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 06:06:21.177154 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3686,"thread_start_us":127,"threads_started":1}
I20260808 06:06:21.178300 20185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:06:21.165008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42920 (local address 127.1.151.129:42749)
0808 06:06:21.169392 (+ 4384us) server_negotiation.cc:206] Beginning negotiation
0808 06:06:21.169400 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 06:06:21.169424 (+ 24us) server_negotiation.cc:407] Connection header received
0808 06:06:21.169934 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:06:21.169941 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:06:21.170234 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:06:21.170383 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:06:21.170958 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:06:21.172961 (+ 2003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:06:21.174761 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:06:21.176356 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:06:21.176445 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:06:21.176604 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 06:06:21.178005 (+ 1401us) server_negotiation.cc:299] Negotiation successful
0808 06:06:21.178125 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4266,"thread_start_us":133,"threads_started":1}
I20260808 06:06:21.181867 19657 tablet_service.cc:1467] Processing CreateTablet for tablet 8a37eec4788348eb8b28ca7a36607d4b (DEFAULT_TABLE table=test-table [id=6b9e993f0b734859917faaa46c83c07d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:06:21.183327 19657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a37eec4788348eb8b28ca7a36607d4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:06:21.187479 20189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:06:21.166547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:44359 (local address 127.0.0.1:60196)
0808 06:06:21.168907 (+ 2360us) negotiation.cc:107] Waiting for socket to connect
0808 06:06:21.168921 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:06:21.169086 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:06:21.174985 (+ 5899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:06:21.174988 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:06:21.175005 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:06:21.175102 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:06:21.175108 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:06:21.183785 (+ 8677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:06:21.183789 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:06:21.185076 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:06:21.185089 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:06:21.187049 (+ 1960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:06:21.187053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:06:21.187185 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:06:21.187214 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 06:06:21.187266 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 06:06:21.187337 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2293,"thread_start_us":115,"threads_started":1}
I20260808 06:06:21.193295 20188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:06:21.166729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60196 (local address 127.1.151.130:44359)
0808 06:06:21.174352 (+ 7623us) server_negotiation.cc:206] Beginning negotiation
0808 06:06:21.174359 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 06:06:21.174384 (+ 25us) server_negotiation.cc:407] Connection header received
0808 06:06:21.174457 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:06:21.174462 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:06:21.174671 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:06:21.174832 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:06:21.175236 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:06:21.176323 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:06:21.186024 (+ 9701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:06:21.186899 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:06:21.192809 (+ 5910us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:06:21.192985 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 06:06:21.193039 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 06:06:21.193146 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7508,"thread_start_us":83,"threads_started":1}
I20260808 06:06:21.194080 20192 tablet_bootstrap.cc:492] T 8a37eec4788348eb8b28ca7a36607d4b P 313b40a3a8d84c63ba2dab93cc69b540: Bootstrap starting.
I20260808 06:06:21.196749 19867 tablet_service.cc:1467] Processing CreateTablet for tablet 8a37eec4788348eb8b28ca7a36607d4b (DEFAULT_TABLE table=test-table [id=6b9e993f0b734859917faaa46c83c07d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:06:21.197539 20192 tablet_bootstrap.cc:654] T 8a37eec4788348eb8b28ca7a36607d4b P 313b40a3a8d84c63ba2dab93cc69b540: Neither blocks nor log segments found. Creating new log.
I20260808 06:06:21.197970 19867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a37eec4788348eb8b28ca7a36607d4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:06:21.198345 20187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:06:21.167250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:41971 (local address 127.0.0.1:50752)
0808 06:06:21.169584 (+ 2334us) negotiation.cc:107] Waiting for socket to connect
0808 06:06:21.169597 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:06:21.169695 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:06:21.177218 (+ 7523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:06:21.177221 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:06:21.177237 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:06:21.177333 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:06:21.177338 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:06:21.193675 (+ 16337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:06:21.193680 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:06:21.194885 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:06:21.194898 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:06:21.197898 (+ 3000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:06:21.197903 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:06:21.198004 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:06:21.198047 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 06:06:21.198103 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 06:06:21.198205 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2274,"thread_start_us":102,"threads_started":1}
I20260808 06:06:21.198603 20192 log.cc:824] T 8a37eec4788348eb8b28ca7a36607d4b P 313b40a3a8d84c63ba2dab93cc69b540: Log is configured to *not* fsync() on all Append() calls
I20260808 06:06:21.199347 20190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:06:21.167633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50752 (local address 127.1.151.131:41971)
0808 06:06:21.176621 (+ 8988us) server_negotiation.cc:206] Beginning negotiation
0808 06:06:21.176634 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 06:06:21.176656 (+ 22us) server_negotiation.cc:407] Connection header received
0808 06:06:21.176726 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:06:21.176731 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:06:21.176928 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:06:21.177077 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:06:21.182262 (+ 5185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:06:21.183623 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:06:21.196875 (+ 13252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:06:21.197730 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:06:21.198815 (+ 1085us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:06:21.198974 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 06:06:21.199043 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 06:06:21.199141 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8865,"thread_start_us":154,"threads_started":1}
I20260808 06:06:21.202275 20192 tablet_bootstrap.cc:492] T 8a37eec4788348eb8b28ca7a36607d4b P 313b40a3a8d84c63ba2dab93cc69b540: No bootstrap required, opened a new log
I20260808 06:06:21.202543 20192 ts_tablet_manager.cc:1397] T 8a37eec4788348eb8b28ca7a36607d4b P 313b40a3a8d84c63ba2dab93cc69b540: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.002s
I20260808 06:06:21.202414 20074 tablet_service.cc:1467] Processing CreateTablet for tablet 8a37eec4788348eb8b28ca7a36607d4b (DEFAULT_TABLE table=test-table [id=6b9e993f0b734859917faaa46c83c07d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:06:21.203519 20074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a37eec4788348eb8b28ca7a36607d4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:06:21.208549 20199 tablet_bootstrap.cc:492] T 8a37eec4788348eb8b28ca7a36607d4b P bb9051f3cd1543738bbbda9fa1d0b8e7: Bootstrap starting.
I20260808 06:06:21.208907 20192 raft_consensus.cc:348] T 8a37eec4788348eb8b28ca7a36607d4b P 313b40a3a8d84c63ba2dab93cc69b540 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313b40a3a8d84c63ba2dab93cc69b540" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 42749 } } peers { permanent_uuid: "bb9051f3cd1543738bbbda9fa1d0b8e7" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44359 } } peers { permanent_uuid: "917d9c3aff9942f483b4befb516f1ab0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 41971 } }
I20260808 06:06:21.209275 20192 raft_consensus.cc:374] T 8a37eec4788348eb8b28ca7a36607d4b P 313b40a3a8d84c63ba2dab93cc69b540 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:06:21.209333 20192 raft_consensus.cc:729] T 8a37eec4788348eb8b28ca7a36607d4b P 313b40a3a8d84c63ba2dab93cc69b540 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 313b40a3a8d84c63ba2dab93cc69b540, State: Initialized, Role: FOLLOWER
I20260808 06:06:21.210003 20192 consensus_queue.cc:260] T 8a37eec4788348eb8b28ca7a36607d4b P 313b40a3a8d84c63ba2dab93cc69b540 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313b40a3a8d84c63ba2dab93cc69b540" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 42749 } } peers { permanent_uuid: "bb9051f3cd1543738bbbda9fa1d0b8e7" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44359 } } peers { permanent_uuid: "917d9c3aff9942f483b4befb516f1ab0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 41971 } }
I20260808 06:06:21.211388 20192 ts_tablet_manager.cc:1428] T 8a37eec4788348eb8b28ca7a36607d4b P 313b40a3a8d84c63ba2dab93cc69b540: Time spent starting tablet: real 0.009s user 0.003s sys 0.004s
I20260808 06:06:21.211874 19754 heartbeater.cc:503] Master 127.1.151.190:45561 was elected leader, sending a full tablet report...
I20260808 06:06:21.213071 20201 tablet_bootstrap.cc:492] T 8a37eec4788348eb8b28ca7a36607d4b P 917d9c3aff9942f483b4befb516f1ab0: Bootstrap starting.
I20260808 06:06:21.213889 20199 tablet_bootstrap.cc:654] T 8a37eec4788348eb8b28ca7a36607d4b P bb9051f3cd1543738bbbda9fa1d0b8e7: Neither blocks nor log segments found. Creating new log.
I20260808 06:06:21.214963 20199 log.cc:824] T 8a37eec4788348eb8b28ca7a36607d4b P bb9051f3cd1543738bbbda9fa1d0b8e7: Log is configured to *not* fsync() on all Append() calls
I20260808 06:06:21.216347 20201 tablet_bootstrap.cc:654] T 8a37eec4788348eb8b28ca7a36607d4b P 917d9c3aff9942f483b4befb516f1ab0: Neither blocks nor log segments found. Creating new log.
I20260808 06:06:21.217291 20199 tablet_bootstrap.cc:492] T 8a37eec4788348eb8b28ca7a36607d4b P bb9051f3cd1543738bbbda9fa1d0b8e7: No bootstrap required, opened a new log
I20260808 06:06:21.217450 20201 log.cc:824] T 8a37eec4788348eb8b28ca7a36607d4b P 917d9c3aff9942f483b4befb516f1ab0: Log is configured to *not* fsync() on all Append() calls
I20260808 06:06:21.217505 20199 ts_tablet_manager.cc:1397] T 8a37eec4788348eb8b28ca7a36607d4b P bb9051f3cd1543738bbbda9fa1d0b8e7: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260808 06:06:21.220002 20201 tablet_bootstrap.cc:492] T 8a37eec4788348eb8b28ca7a36607d4b P 917d9c3aff9942f483b4befb516f1ab0: No bootstrap required, opened a new log
I20260808 06:06:21.220204 20201 ts_tablet_manager.cc:1397] T 8a37eec4788348eb8b28ca7a36607d4b P 917d9c3aff9942f483b4befb516f1ab0: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 06:06:21.223721 20199 raft_consensus.cc:348] T 8a37eec4788348eb8b28ca7a36607d4b P bb9051f3cd1543738bbbda9fa1d0b8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313b40a3a8d84c63ba2dab93cc69b540" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 42749 } } peers { permanent_uuid: "bb9051f3cd1543738bbbda9fa1d0b8e7" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44359 } } peers { permanent_uuid: "917d9c3aff9942f483b4befb516f1ab0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 41971 } }
I20260808 06:06:21.224063 20199 raft_consensus.cc:374] T 8a37eec4788348eb8b28ca7a36607d4b P bb9051f3cd1543738bbbda9fa1d0b8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:06:21.224171 20199 raft_consensus.cc:729] T 8a37eec4788348eb8b28ca7a36607d4b P bb9051f3cd1543738bbbda9fa1d0b8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb9051f3cd1543738bbbda9fa1d0b8e7, State: Initialized, Role: FOLLOWER
I20260808 06:06:21.224716 20199 consensus_queue.cc:260] T 8a37eec4788348eb8b28ca7a36607d4b P bb9051f3cd1543738bbbda9fa1d0b8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313b40a3a8d84c63ba2dab93cc69b540" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 42749 } } peers { permanent_uuid: "bb9051f3cd1543738bbbda9fa1d0b8e7" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44359 } } peers { permanent_uuid: "917d9c3aff9942f483b4befb516f1ab0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 41971 } }
I20260808 06:06:21.224917 20201 raft_consensus.cc:348] T 8a37eec4788348eb8b28ca7a36607d4b P 917d9c3aff9942f483b4befb516f1ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313b40a3a8d84c63ba2dab93cc69b540" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 42749 } } peers { permanent_uuid: "bb9051f3cd1543738bbbda9fa1d0b8e7" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44359 } } peers { permanent_uuid: "917d9c3aff9942f483b4befb516f1ab0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 41971 } }
I20260808 06:06:21.225200 20201 raft_consensus.cc:374] T 8a37eec4788348eb8b28ca7a36607d4b P 917d9c3aff9942f483b4befb516f1ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:06:21.225270 20201 raft_consensus.cc:729] T 8a37eec4788348eb8b28ca7a36607d4b P 917d9c3aff9942f483b4befb516f1ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 917d9c3aff9942f483b4befb516f1ab0, State: Initialized, Role: FOLLOWER
I20260808 06:06:21.225690 20201 consensus_queue.cc:260] T 8a37eec4788348eb8b28ca7a36607d4b P 917d9c3aff9942f483b4befb516f1ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313b40a3a8d84c63ba2dab93cc69b540" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 42749 } } peers { permanent_uuid: "bb9051f3cd1543738bbbda9fa1d0b8e7" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44359 } } peers { permanent_uuid: "917d9c3aff9942f483b4befb516f1ab0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 41971 } }
I20260808 06:06:21.225860 20199 ts_tablet_manager.cc:1428] T 8a37eec4788348eb8b28ca7a36607d4b P bb9051f3cd1543738bbbda9fa1d0b8e7: Time spent starting tablet: real 0.008s user 0.009s sys 0.000s
I20260808 06:06:21.226248 19973 heartbeater.cc:503] Master 127.1.151.190:45561 was elected leader, sending a full tablet report...
I20260808 06:06:21.226576 20170 heartbeater.cc:503] Master 127.1.151.190:45561 was elected leader, sending a full tablet report...
I20260808 06:06:21.226742 20201 ts_tablet_manager.cc:1428] T 8a37eec4788348eb8b28ca7a36607d4b P 917d9c3aff9942f483b4befb516f1ab0: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260808 06:06:21.346715 20171 tablet.cc:2369] T 8a37eec4788348eb8b28ca7a36607d4b P 917d9c3aff9942f483b4befb516f1ab0: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 06:06:21 dist-test-slave-zj8q krb5kdc[18736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169180, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for HTTP/127.1.151.148@KRBTEST.COM
W20260808 06:06:21.376766 19974 tablet.cc:2369] T 8a37eec4788348eb8b28ca7a36607d4b P bb9051f3cd1543738bbbda9fa1d0b8e7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 06:06:21.441336 19756 tablet.cc:2369] T 8a37eec4788348eb8b28ca7a36607d4b P 313b40a3a8d84c63ba2dab93cc69b540: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 06:06:21.499766 20206 raft_consensus.cc:482] T 8a37eec4788348eb8b28ca7a36607d4b P bb9051f3cd1543738bbbda9fa1d0b8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:06:21.500149 20206 raft_consensus.cc:504] T 8a37eec4788348eb8b28ca7a36607d4b P bb9051f3cd1543738bbbda9fa1d0b8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313b40a3a8d84c63ba2dab93cc69b540" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 42749 } } peers { permanent_uuid: "bb9051f3cd1543738bbbda9fa1d0b8e7" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44359 } } peers { permanent_uuid: "917d9c3aff9942f483b4befb516f1ab0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 41971 } }
I20260808 06:06:21.501765 20206 leader_election.cc:288] T 8a37eec4788348eb8b28ca7a36607d4b P bb9051f3cd1543738bbbda9fa1d0b8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 313b40a3a8d84c63ba2dab93cc69b540 (127.1.151.129:42749), 917d9c3aff9942f483b4befb516f1ab0 (127.1.151.131:41971)
I20260808 06:06:21.512382 20190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:06:21.502246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:34029 (local address 127.1.151.131:41971)
0808 06:06:21.502707 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0808 06:06:21.502712 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:06:21.503391 (+ 679us) server_negotiation.cc:407] Connection header received
0808 06:06:21.503610 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:06:21.503613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:06:21.503680 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:06:21.503731 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:06:21.504800 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:06:21.505644 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:06:21.510938 (+ 5294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:06:21.511563 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:06:21.511677 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:06:21.511735 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 06:06:21.512154 (+ 419us) server_negotiation.cc:299] Negotiation successful
0808 06:06:21.512244 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366}
I20260808 06:06:21.513105 20215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:06:21.502121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:41971 (local address 127.1.151.130:34029)
0808 06:06:21.503078 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0808 06:06:21.503090 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 06:06:21.503476 (+ 386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:06:21.504317 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:06:21.504320 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:06:21.504341 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:06:21.504650 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:06:21.504661 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:06:21.509141 (+ 4480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:06:21.509144 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:06:21.510788 (+ 1644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:06:21.510801 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:06:21.511877 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:06:21.511880 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:06:21.512004 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:06:21.512028 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 06:06:21.512813 (+ 785us) client_negotiation.cc:240] Negotiation successful
0808 06:06:21.512897 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":894,"thread_start_us":183,"threads_started":1}
I20260808 06:06:21.514135 20099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a37eec4788348eb8b28ca7a36607d4b" candidate_uuid: "bb9051f3cd1543738bbbda9fa1d0b8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "917d9c3aff9942f483b4befb516f1ab0" is_pre_election: true
I20260808 06:06:21.514510 20099 raft_consensus.cc:2454] T 8a37eec4788348eb8b28ca7a36607d4b P 917d9c3aff9942f483b4befb516f1ab0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb9051f3cd1543738bbbda9fa1d0b8e7 in term 0.
I20260808 06:06:21.515619 19799 leader_election.cc:302] T 8a37eec4788348eb8b28ca7a36607d4b P bb9051f3cd1543738bbbda9fa1d0b8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 917d9c3aff9942f483b4befb516f1ab0, bb9051f3cd1543738bbbda9fa1d0b8e7; no voters:
I20260808 06:06:21.515945 20206 raft_consensus.cc:2781] T 8a37eec4788348eb8b28ca7a36607d4b P bb9051f3cd1543738bbbda9fa1d0b8e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 06:06:21.516016 20206 raft_consensus.cc:482] T 8a37eec4788348eb8b28ca7a36607d4b P bb9051f3cd1543738bbbda9fa1d0b8e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 06:06:21.516065 20206 raft_consensus.cc:3037] T 8a37eec4788348eb8b28ca7a36607d4b P bb9051f3cd1543738bbbda9fa1d0b8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:06:21.517576 20185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:06:21.502684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:56935 (local address 127.1.151.129:42749)
0808 06:06:21.502821 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 06:06:21.502825 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:06:21.502841 (+ 16us) server_negotiation.cc:407] Connection header received
0808 06:06:21.502896 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:06:21.502900 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:06:21.502952 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:06:21.503033 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:06:21.513874 (+ 10841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:06:21.514679 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:06:21.516249 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:06:21.516874 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:06:21.516973 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:06:21.517022 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 06:06:21.517376 (+ 354us) server_negotiation.cc:299] Negotiation successful
0808 06:06:21.517458 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 06:06:21.517954 20213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:06:21.502051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:42749 (local address 127.1.151.130:56935)
0808 06:06:21.502633 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0808 06:06:21.502659 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 06:06:21.502810 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:06:21.513467 (+ 10657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:06:21.513470 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:06:21.513490 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:06:21.513711 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:06:21.513722 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:06:21.514832 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:06:21.514835 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:06:21.516095 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:06:21.516108 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:06:21.517120 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:06:21.517124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:06:21.517229 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:06:21.517253 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 06:06:21.517735 (+ 482us) client_negotiation.cc:240] Negotiation successful
0808 06:06:21.517813 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":139,"threads_started":1}
I20260808 06:06:21.518239 20206 raft_consensus.cc:504] T 8a37eec4788348eb8b28ca7a36607d4b P bb9051f3cd1543738bbbda9fa1d0b8e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313b40a3a8d84c63ba2dab93cc69b540" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 42749 } } peers { permanent_uuid: "bb9051f3cd1543738bbbda9fa1d0b8e7" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44359 } } peers { permanent_uuid: "917d9c3aff9942f483b4befb516f1ab0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 41971 } }
I20260808 06:06:21.518601 20206 leader_election.cc:288] T 8a37eec4788348eb8b28ca7a36607d4b P bb9051f3cd1543738bbbda9fa1d0b8e7 [CANDIDATE]: Term 1 election: Requested vote from peers 313b40a3a8d84c63ba2dab93cc69b540 (127.1.151.129:42749), 917d9c3aff9942f483b4befb516f1ab0 (127.1.151.131:41971)
I20260808 06:06:21.518997 19685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a37eec4788348eb8b28ca7a36607d4b" candidate_uuid: "bb9051f3cd1543738bbbda9fa1d0b8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "313b40a3a8d84c63ba2dab93cc69b540" is_pre_election: true
I20260808 06:06:21.519304 19685 raft_consensus.cc:2454] T 8a37eec4788348eb8b28ca7a36607d4b P 313b40a3a8d84c63ba2dab93cc69b540 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb9051f3cd1543738bbbda9fa1d0b8e7 in term 0.
I20260808 06:06:21.520838 19685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a37eec4788348eb8b28ca7a36607d4b" candidate_uuid: "bb9051f3cd1543738bbbda9fa1d0b8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "313b40a3a8d84c63ba2dab93cc69b540"
I20260808 06:06:21.520951 19685 raft_consensus.cc:3037] T 8a37eec4788348eb8b28ca7a36607d4b P 313b40a3a8d84c63ba2dab93cc69b540 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:06:21.522433 20099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a37eec4788348eb8b28ca7a36607d4b" candidate_uuid: "bb9051f3cd1543738bbbda9fa1d0b8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "917d9c3aff9942f483b4befb516f1ab0"
I20260808 06:06:21.522562 20099 raft_consensus.cc:3037] T 8a37eec4788348eb8b28ca7a36607d4b P 917d9c3aff9942f483b4befb516f1ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:06:21.525727 20099 raft_consensus.cc:2454] T 8a37eec4788348eb8b28ca7a36607d4b P 917d9c3aff9942f483b4befb516f1ab0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb9051f3cd1543738bbbda9fa1d0b8e7 in term 1.
I20260808 06:06:21.526449 19685 raft_consensus.cc:2454] T 8a37eec4788348eb8b28ca7a36607d4b P 313b40a3a8d84c63ba2dab93cc69b540 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb9051f3cd1543738bbbda9fa1d0b8e7 in term 1.
I20260808 06:06:21.526887 19800 leader_election.cc:302] T 8a37eec4788348eb8b28ca7a36607d4b P bb9051f3cd1543738bbbda9fa1d0b8e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 313b40a3a8d84c63ba2dab93cc69b540, bb9051f3cd1543738bbbda9fa1d0b8e7; no voters:
I20260808 06:06:21.527153 20206 raft_consensus.cc:2781] T 8a37eec4788348eb8b28ca7a36607d4b P bb9051f3cd1543738bbbda9fa1d0b8e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:06:21.527371 20206 raft_consensus.cc:686] T 8a37eec4788348eb8b28ca7a36607d4b P bb9051f3cd1543738bbbda9fa1d0b8e7 [term 1 LEADER]: Becoming Leader. State: Replica: bb9051f3cd1543738bbbda9fa1d0b8e7, State: Running, Role: LEADER
I20260808 06:06:21.527724 20206 consensus_queue.cc:237] T 8a37eec4788348eb8b28ca7a36607d4b P bb9051f3cd1543738bbbda9fa1d0b8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313b40a3a8d84c63ba2dab93cc69b540" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 42749 } } peers { permanent_uuid: "bb9051f3cd1543738bbbda9fa1d0b8e7" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44359 } } peers { permanent_uuid: "917d9c3aff9942f483b4befb516f1ab0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 41971 } }
I20260808 06:06:21.534283 19438 catalog_manager.cc:5495] T 8a37eec4788348eb8b28ca7a36607d4b P bb9051f3cd1543738bbbda9fa1d0b8e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to bb9051f3cd1543738bbbda9fa1d0b8e7 (127.1.151.130). New cstate: current_term: 1 leader_uuid: "bb9051f3cd1543738bbbda9fa1d0b8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313b40a3a8d84c63ba2dab93cc69b540" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 42749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb9051f3cd1543738bbbda9fa1d0b8e7" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "917d9c3aff9942f483b4befb516f1ab0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 41971 } health_report { overall_health: UNKNOWN } } }
I20260808 06:06:21.636945 20188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:06:21.625641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60200 (local address 127.1.151.130:44359)
0808 06:06:21.626107 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0808 06:06:21.626112 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:06:21.626127 (+ 15us) server_negotiation.cc:407] Connection header received
0808 06:06:21.626260 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:06:21.626263 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:06:21.626341 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:06:21.626457 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:06:21.626943 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:06:21.627785 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:06:21.634342 (+ 6557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:06:21.634718 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:06:21.634838 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:06:21.635308 (+ 470us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:06:21.635884 (+ 576us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:06:21.636006 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 06:06:21.636727 (+ 721us) server_negotiation.cc:299] Negotiation successful
0808 06:06:21.636803 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367}
I20260808 06:06:21.650192 20099 raft_consensus.cc:1261] T 8a37eec4788348eb8b28ca7a36607d4b P 917d9c3aff9942f483b4befb516f1ab0 [term 1 FOLLOWER]: Refusing update from remote peer bb9051f3cd1543738bbbda9fa1d0b8e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:06:21.651263 20216 consensus_queue.cc:1035] T 8a37eec4788348eb8b28ca7a36607d4b P bb9051f3cd1543738bbbda9fa1d0b8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "917d9c3aff9942f483b4befb516f1ab0" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 41971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:06:21.659188 19685 raft_consensus.cc:1261] T 8a37eec4788348eb8b28ca7a36607d4b P 313b40a3a8d84c63ba2dab93cc69b540 [term 1 FOLLOWER]: Refusing update from remote peer bb9051f3cd1543738bbbda9fa1d0b8e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:06:21.659993 20216 consensus_queue.cc:1035] T 8a37eec4788348eb8b28ca7a36607d4b P bb9051f3cd1543738bbbda9fa1d0b8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "313b40a3a8d84c63ba2dab93cc69b540" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 42749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:06:21.688287 20223 mvcc.cc:205] Tried to move back new op lower bound from 7316148968030789632 to 7316148967540629504. Current Snapshot: MvccSnapshot[applied={T|T < 7316148968030789632}]
I20260808 06:06:21.710083 19438 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60550:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:06:21.710323 19438 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60550:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:06:21.713692 19438 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ae702375cd0c41f5be2b5b4baa7f37a6: Sending DeleteTablet for 3 replicas of tablet 8a37eec4788348eb8b28ca7a36607d4b
I20260808 06:06:21.714663 19867 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a37eec4788348eb8b28ca7a36607d4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:06:21 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:60196
I20260808 06:06:21.714725 20074 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a37eec4788348eb8b28ca7a36607d4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:06:21 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:50752
I20260808 06:06:21.715729 19657 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a37eec4788348eb8b28ca7a36607d4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:06:21 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:42920
I20260808 06:06:21.716974 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 19562
I20260808 06:06:21.738186 20248 tablet_replica.cc:330] T 8a37eec4788348eb8b28ca7a36607d4b P bb9051f3cd1543738bbbda9fa1d0b8e7: stopping tablet replica
I20260808 06:06:21.738695 20248 raft_consensus.cc:2229] T 8a37eec4788348eb8b28ca7a36607d4b P bb9051f3cd1543738bbbda9fa1d0b8e7 [term 1 LEADER]: Raft consensus shutting down.
I20260808 06:06:21.739338 20248 raft_consensus.cc:2258] T 8a37eec4788348eb8b28ca7a36607d4b P bb9051f3cd1543738bbbda9fa1d0b8e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:06:21.741106 20248 ts_tablet_manager.cc:1905] T 8a37eec4788348eb8b28ca7a36607d4b P bb9051f3cd1543738bbbda9fa1d0b8e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:06:21.744272 20248 ts_tablet_manager.cc:1918] T 8a37eec4788348eb8b28ca7a36607d4b P bb9051f3cd1543738bbbda9fa1d0b8e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:06:21.744457 20248 log.cc:1196] T 8a37eec4788348eb8b28ca7a36607d4b P bb9051f3cd1543738bbbda9fa1d0b8e7: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-1/wal/wals/8a37eec4788348eb8b28ca7a36607d4b
I20260808 06:06:21.744890 20248 ts_tablet_manager.cc:1939] T 8a37eec4788348eb8b28ca7a36607d4b P bb9051f3cd1543738bbbda9fa1d0b8e7: Deleting consensus metadata
I20260808 06:06:21.745383 20252 tablet_replica.cc:330] T 8a37eec4788348eb8b28ca7a36607d4b P 917d9c3aff9942f483b4befb516f1ab0: stopping tablet replica
I20260808 06:06:21.745865 20252 raft_consensus.cc:2229] T 8a37eec4788348eb8b28ca7a36607d4b P 917d9c3aff9942f483b4befb516f1ab0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:06:21.747439 19423 catalog_manager.cc:4841] TS bb9051f3cd1543738bbbda9fa1d0b8e7 (127.1.151.130:44359): tablet 8a37eec4788348eb8b28ca7a36607d4b (table test-table [id=6b9e993f0b734859917faaa46c83c07d]) successfully deleted
I20260808 06:06:21.748095 20252 raft_consensus.cc:2258] T 8a37eec4788348eb8b28ca7a36607d4b P 917d9c3aff9942f483b4befb516f1ab0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 06:06:21.748569 19422 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.151.129:42749 (error 108)
I20260808 06:06:21.749833 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 19767
I20260808 06:06:21.753545 20252 ts_tablet_manager.cc:1905] T 8a37eec4788348eb8b28ca7a36607d4b P 917d9c3aff9942f483b4befb516f1ab0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:06:21.756310 20252 ts_tablet_manager.cc:1918] T 8a37eec4788348eb8b28ca7a36607d4b P 917d9c3aff9942f483b4befb516f1ab0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:06:21.756415 20252 log.cc:1196] T 8a37eec4788348eb8b28ca7a36607d4b P 917d9c3aff9942f483b4befb516f1ab0: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786169070948234-1630-0/minicluster-data/ts-2/wal/wals/8a37eec4788348eb8b28ca7a36607d4b
I20260808 06:06:21.756857 20252 ts_tablet_manager.cc:1939] T 8a37eec4788348eb8b28ca7a36607d4b P 917d9c3aff9942f483b4befb516f1ab0: Deleting consensus metadata
I20260808 06:06:21.758359 19421 catalog_manager.cc:4841] TS 917d9c3aff9942f483b4befb516f1ab0 (127.1.151.131:41971): tablet 8a37eec4788348eb8b28ca7a36607d4b (table test-table [id=6b9e993f0b734859917faaa46c83c07d]) successfully deleted
I20260808 06:06:21.762856 20256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:06:21.758902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:42749 (local address 127.0.0.1:42934)
0808 06:06:21.762638 (+ 3736us) negotiation.cc:107] Waiting for socket to connect
0808 06:06:21.762733 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.129:42749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3634,"thread_start_us":84,"threads_started":1}
W20260808 06:06:21.763144 19422 catalog_manager.cc:4584] TS 313b40a3a8d84c63ba2dab93cc69b540 (127.1.151.129:42749): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8a37eec4788348eb8b28ca7a36607d4b: Network error: Client connection negotiation failed: client connection to 127.1.151.129:42749: connect: Connection refused (error 111)
I20260808 06:06:21.764321 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 19979
I20260808 06:06:21.783115 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 19394
I20260808 06:06:21.796761 1630 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 06:06:22.142879 1630 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 06:06:22.143005 1630 mini_ranger.cc:66] Stopping Ranger...
I20260808 06:06:22.487143 1630 mini_ranger.cc:68] Stopped Ranger
2026-08-08 06:06:22.487 UTC [18775] LOG: received smart shutdown request
2026-08-08 06:06:22.491 UTC [18775] LOG: background worker "logical replication launcher" (PID 18786) exited with exit code 1
2026-08-08 06:06:22.491 UTC [18780] LOG: shutting down
2026-08-08 06:06:22.642 UTC [18775] LOG: database system is shut down
2026-08-08T06:06:22Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48403 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:06:22 dist-test-slave-zj8q krb5kdc[20271](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:06:22 dist-test-slave-zj8q krb5kdc[20271](info): set up 2 sockets
Aug 08 06:06:22 dist-test-slave-zj8q krb5kdc[20271](info): commencing operation
krb5kdc: starting...
W20260808 06:06:23.925866 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:06:24 dist-test-slave-zj8q krb5kdc[20271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:06:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:06:24Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.151.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.151.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.151.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/rangeradmin_127.1.151.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.151.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/rangeradmin_127.1.151.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.151.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.151.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.151.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/rangerlookup_127.1.151.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.151.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/rangerlookup_127.1.151.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.151.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.151.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.151.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/HTTP_127.1.151.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.151.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/HTTP_127.1.151.148@KRBTEST.COM.keytab.
I20260808 06:06:24.090253 1630 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskIVYy3F/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/postgres -l logfile start
2026-08-08 06:06:26.362 UTC [20315] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 06:06:26.362 UTC [20315] LOG: listening on IPv4 address "127.1.151.148", port 34965
2026-08-08 06:06:26.365 UTC [20315] LOG: listening on Unix socket "/tmp/.s.PGSQL.34965"
2026-08-08 06:06:26.384 UTC [20321] LOG: database system was shut down at 2026-08-08 06:06:25 UTC
2026-08-08 06:06:26.388 UTC [20315] LOG: database system is ready to accept connections
I20260808 06:06:27.441923 1630 mini_postgres.cc:96] Postgres bound to 34965
2026-08-08 06:06:27.449 UTC [20335] FATAL: database "slave" does not exist
127.1.151.148:34965 - accepting connections
I20260808 06:06:27.450465 1630 mini_ranger.cc:161] Starting Ranger...
I20260808 06:06:27.465961 1630 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 06:06:27.911422 1630 mini_ranger.cc:87] Created ranger Postgres database
I20260808 06:06:27.911499 1630 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ranger-admin
2026-08-08 06:06:28,175 [I] DB FLAVOR :POSTGRES
2026-08-08 06:06:28,176 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 06:06:28,176 [I] Checking connection..
2026-08-08 06:06:28,176 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:34965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 06:06:28,420 [I] Checking connection passed.
2026-08-08 06:06:28,420 [I] --------- Verifying version history table ---------
2026-08-08 06:06:28,420 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:34965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 06:06:28,679 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 06:06:28,679 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:34965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 06:06:28,914 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 06:06:28,914 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 06:06:28,915 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:34965/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 06:06:29.164 UTC [20435] WARNING: there is no transaction in progress
2026-08-08 06:06:29,172 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 06:06:29,172 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:34965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 06:06:29,431 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 06:06:29,431 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 06:06:29,431 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:34965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 06:06:29,677 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:34965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 06:06:29,921 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:34965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-zj8q.c.gcp-upstream.internal','N') ;"
2026-08-08 06:06:30,160 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:34965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 06:06:30,411 [I] Table x_portal_user does not exist in database ranger
2026-08-08 06:06:30,411 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:34965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 06:06:30,643 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 06:06:30,644 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:34965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 06:06:30,878 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:34965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 06:06:31,137 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 06:06:31,137 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:34965/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 06:06:32.045 UTC [20604] WARNING: there is no transaction in progress
2026-08-08 06:06:32.060 UTC [20604] WARNING: there is no transaction in progress
2026-08-08 06:06:32.075 UTC [20604] WARNING: there is no transaction in progress
2026-08-08 06:06:32.088 UTC [20604] WARNING: there is no transaction in progress
2026-08-08 06:06:32.102 UTC [20604] WARNING: there is no transaction in progress
2026-08-08 06:06:32.119 UTC [20604] WARNING: there is no transaction in progress
2026-08-08 06:06:32.170 UTC [20604] WARNING: there is no transaction in progress
2026-08-08 06:06:32.177 UTC [20604] WARNING: there is no transaction in progress
2026-08-08 06:06:32.190 UTC [20604] WARNING: there is no transaction in progress
2026-08-08 06:06:32.200 UTC [20604] WARNING: there is no transaction in progress
2026-08-08 06:06:32.209 UTC [20604] WARNING: there is no transaction in progress
2026-08-08 06:06:32.222 UTC [20604] WARNING: there is no transaction in progress
2026-08-08 06:06:32.231 UTC [20604] WARNING: there is no transaction in progress
2026-08-08 06:06:32.236 UTC [20604] WARNING: there is no transaction in progress
2026-08-08 06:06:32.242 UTC [20604] WARNING: there is no transaction in progress
2026-08-08 06:06:32.248 UTC [20604] WARNING: there is no transaction in progress
2026-08-08 06:06:32,809 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 06:06:32,809 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:34965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 06:06:33,050 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 06:06:33,050 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:34965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 06:06:33,307 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 06:06:33,307 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:34965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 06:06:33,554 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 06:06:33,555 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:34965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 06:06:33,800 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:34965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 06:06:34,053 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:34965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-zj8q.c.gcp-upstream.internal';"
2026-08-08 06:06:34,298 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 06:06:34,299 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:34965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 06:06:34,547 [I] DB_PATCHES have already been applied
I20260808 06:06:34.553568 1630 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ranger-admin:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskIVYy3F/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 06:06:34.553663 1630 mini_ranger.cc:193] Using host: 127.1.151.148
I20260808 06:06:34.557896 1630 mini_ranger.cc:227] Ranger admin URL: http://127.1.151.148:58233
Aug 08, 2026 6:06:34 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 6:06:34 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 6:06:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 6:06:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 6:06:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 6:06:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 6:06:35 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 06:06:35.793342 1630 mini_ranger.cc:160] Time spent starting Ranger: real 8.343s user 0.000s sys 0.011s
Aug 08, 2026 6:06:55 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 6:06:55 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 6:06:55 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 6:06:55 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 6:06:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 6:06:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 6:06:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 6:06:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 6:06:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 6:06:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 6:06:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 6:06:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 6:06:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 6:06:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 6:06:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 6:06:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 6:06:55 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 6:06:56 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 06:06:57.019829 1630 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.151.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.151.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.151.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/rangerkms_127.1.151.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.151.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/rangerkms_127.1.151.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.151.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.151.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.151.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/HTTP_127.1.151.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.151.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/HTTP_127.1.151.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 06:06:57 dist-test-slave-zj8q krb5kdc[20271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169217, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 06:06:57.101709 1630 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 06:06:57.111564 1630 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 06:06:57.842705 1630 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 06:06:57.842795 1630 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ranger-kms
I20260808 06:06:57.842813 1630 mini_ranger_kms.cc:227] Using postgres at 127.1.151.148:34965
2026-08-08 06:06:58,001 [I] DB FLAVOR :POSTGRES
2026-08-08 06:06:58,001 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 06:06:58,001 [I] Checking connection
2026-08-08 06:06:58,001 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:34965/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 06:06:58,232 [I] connection success
2026-08-08 06:06:58,232 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 06:06:58,232 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 06:06:58,232 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:34965/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 06:06:58,457 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 06:06:58,457 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 06:06:58,457 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 06:06:58,457 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.151.148:34965/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 06:06:58,711 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 06:06:59.139074 1630 mini_ranger_kms.cc:310] Created kms service
I20260808 06:06:59.322103 1630 mini_ranger_kms.cc:326] Created kudu user
I20260808 06:06:59.366041 1630 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 06:06:59 dist-test-slave-zj8q krb5kdc[20271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169217, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.151.148@KRBTEST.COM
I20260808 06:06:59.737605 1630 mini_ranger_kms.cc:384] Added ranger policy
I20260808 06:06:59.737756 1630 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ranger-kms:/tmp/dist-test-taskIVYy3F/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskIVYy3F/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 06:06:59.737780 1630 mini_ranger_kms.cc:240] Using host: 127.1.151.148
I20260808 06:06:59.742475 1630 mini_ranger_kms.cc:276] Ranger KMS PID: 20895
I20260808 06:06:59.742558 1630 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.151.148:48363
Aug 08, 2026 6:07:00 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 6:07:00 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 6:07:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 6:07:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 6:07:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 6:07:01 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 06:07:01 dist-test-slave-zj8q krb5kdc[20271](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786169221, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.151.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:07:02.205127 1630 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.103s user 0.013s sys 0.131s
I20260808 06:07:02.205250 1630 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 06:07:02.205327 1630 mini_ranger_kms.cc:401] 127.1.151.148:48363/kms/v1/keys
Aug 08 06:07:02 dist-test-slave-zj8q krb5kdc[20271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169221, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.151.148@KRBTEST.COM for HTTP/127.1.151.148@KRBTEST.COM
Aug 08 06:07:02 dist-test-slave-zj8q krb5kdc[20271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169221, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.151.148@KRBTEST.COM for HTTP/127.1.151.148@KRBTEST.COM
Aug 08 06:07:04 dist-test-slave-zj8q krb5kdc[20271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.151.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
I20260808 06:07:04.975214 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:44349
--webserver_interface=127.1.151.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:40343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:44349
--ranger_config_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.151.148:48363/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:07:05.085318 20980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:07:05.085580 20980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:07:05.085620 20980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:07:05.089550 20980 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:07:05.089607 20980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:07:05.089624 20980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:07:05.089639 20980 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:07:05.089653 20980 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:07:05.095002 20980 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:40343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.151.148:48363/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:44349
--ranger_config_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:44349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20980
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:07:05.096024 20980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:07:05.097013 20980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:07:05.105746 20996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:05.105762 20992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:05.105921 20980 server_base.cc:1034] running on GCE node
W20260808 06:07:05.105741 20991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:05.106626 20980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:07:05.107684 20980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:07:05.108870 20980 hybrid_clock.cc:648] HybridClock initialized: now 1786169225108851 us; error 27 us; skew 500 ppm
Aug 08 06:07:05 dist-test-slave-zj8q krb5kdc[20271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169225, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:07:05.114945 20980 init.cc:377] Logged in from keytab as kudu/127.1.151.190@KRBTEST.COM (short username kudu)
I20260808 06:07:05.115860 20980 webserver.cc:460] Webserver started at http://127.1.151.190:43999/ using document root <none> and password file <none>
I20260808 06:07:05.116389 20980 fs_manager.cc:359] Metadata directory not provided
I20260808 06:07:05.116442 20980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:07:05.116636 20980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 06:07:05 dist-test-slave-zj8q krb5kdc[20271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169225, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for HTTP/127.1.151.148@KRBTEST.COM
I20260808 06:07:05.163080 20980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "3eb419af59bc4da6a8074f1dc3169721"
format_stamp: "Formatted at 2026-08-08 06:07:05 on dist-test-slave-zj8q"
server_key: "90fbab75ac518939fa64f5e6ecf999ea"
server_key_iv: "117311e7459401b7a2ecec7159fcc7e1"
server_key_version: "kuduclusterkey@0"
I20260808 06:07:05.163761 20980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "3eb419af59bc4da6a8074f1dc3169721"
format_stamp: "Formatted at 2026-08-08 06:07:05 on dist-test-slave-zj8q"
server_key: "90fbab75ac518939fa64f5e6ecf999ea"
server_key_iv: "117311e7459401b7a2ecec7159fcc7e1"
server_key_version: "kuduclusterkey@0"
I20260808 06:07:05.167629 20980 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 06:07:05.182212 21009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:05.183498 20980 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 06:07:05.183614 20980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "3eb419af59bc4da6a8074f1dc3169721"
format_stamp: "Formatted at 2026-08-08 06:07:05 on dist-test-slave-zj8q"
server_key: "90fbab75ac518939fa64f5e6ecf999ea"
server_key_iv: "117311e7459401b7a2ecec7159fcc7e1"
server_key_version: "kuduclusterkey@0"
I20260808 06:07:05.183739 20980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:07:05.207180 20980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:07:05.209970 20980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:07:05.210191 20980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:07:05.217909 20980 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:44349
I20260808 06:07:05.217928 21093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:44349 every 8 connection(s)
I20260808 06:07:05.218911 20980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:07:05.221634 21095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:07:05.222397 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 20980
I20260808 06:07:05.222533 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:07:05.222846 1630 external_mini_cluster.cc:1428] Setting key bad1815f867ba313d04edfccc6d3b3c0
I20260808 06:07:05.228111 21095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721: Bootstrap starting.
I20260808 06:07:05.230532 21095 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721: Neither blocks nor log segments found. Creating new log.
I20260808 06:07:05.231381 21095 log.cc:824] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721: Log is configured to *not* fsync() on all Append() calls
Aug 08 06:07:05 dist-test-slave-zj8q krb5kdc[20271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:07:05.233335 21095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721: No bootstrap required, opened a new log
I20260808 06:07:05.237172 21095 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eb419af59bc4da6a8074f1dc3169721" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44349 } }
I20260808 06:07:05.237411 21095 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:07:05.237509 21095 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb419af59bc4da6a8074f1dc3169721, State: Initialized, Role: FOLLOWER
I20260808 06:07:05.237987 21095 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eb419af59bc4da6a8074f1dc3169721" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44349 } }
I20260808 06:07:05.238140 21095 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:07:05.238240 21095 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:07:05.238358 21095 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:07:05.239403 21095 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eb419af59bc4da6a8074f1dc3169721" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44349 } }
I20260808 06:07:05.239765 21095 leader_election.cc:302] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3eb419af59bc4da6a8074f1dc3169721; no voters:
I20260808 06:07:05.239822 21102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:05.224324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60306 (local address 127.1.151.190:44349)
0808 06:07:05.224779 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:05.224797 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:05.224841 (+ 44us) server_negotiation.cc:407] Connection header received
0808 06:07:05.225676 (+ 835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:05.225700 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:05.226086 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:05.226466 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:05.227387 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:05.228787 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:05.229995 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:05.230717 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:05.230877 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:05.233744 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:05.233750 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:05.233765 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:05.233794 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:05.236361 (+ 2567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:05.236863 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:05.236869 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:05.236875 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:05.237001 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:05.237322 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:05.237325 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:05.237327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:05.237813 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:05.238002 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:05.238610 (+ 608us) server_negotiation.cc:299] Negotiation successful
0808 06:07:05.238881 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":134,"threads_started":1}
I20260808 06:07:05.240092 21095 leader_election.cc:288] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:07:05.240453 21107 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:07:05.240708 21107 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721 [term 1 LEADER]: Becoming Leader. State: Replica: 3eb419af59bc4da6a8074f1dc3169721, State: Running, Role: LEADER
I20260808 06:07:05.241086 21107 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eb419af59bc4da6a8074f1dc3169721" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44349 } }
I20260808 06:07:05.241312 21095 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:07:05.242400 21107 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3eb419af59bc4da6a8074f1dc3169721" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eb419af59bc4da6a8074f1dc3169721" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44349 } } }
I20260808 06:07:05.242589 21107 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721 [sys.catalog]: This master's current role is: LEADER
I20260808 06:07:05.242776 21106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3eb419af59bc4da6a8074f1dc3169721. Latest consensus state: current_term: 1 leader_uuid: "3eb419af59bc4da6a8074f1dc3169721" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eb419af59bc4da6a8074f1dc3169721" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44349 } } }
I20260808 06:07:05.242939 21106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721 [sys.catalog]: This master's current role is: LEADER
I20260808 06:07:05.243533 21095 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 06:07:05.247026 21118 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:07:05.251158 21118 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:07:05.259948 21118 catalog_manager.cc:1347] Generated new cluster ID: 522cec458e1d4d32ac67edfd42e16e9c
I20260808 06:07:05.260105 21118 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:07:05.287147 21118 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:07:05.288674 21118 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:07:05.303148 21118 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721: Generated new TSK 0
I20260808 06:07:05.304095 21118 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.151.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab.
Entry for principal kudu/127.1.151.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab.
I20260808 06:07:05.972980 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.151.148:48363/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.129:0
--local_ip_for_outbound_sockets=127.1.151.129
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:44349
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:40343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:07:06.105208 21161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:07:06.105454 21161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:07:06.105495 21161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:07:06.110369 21161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:07:06.110431 21161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:07:06.110496 21161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.129
I20260808 06:07:06.117739 21161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:40343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.151.148:48363/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:44349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.129
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:07:06.118891 21161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:07:06.119951 21161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:07:06.131177 21173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:06.131191 21177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:06.131358 21172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:06.132110 21161 server_base.cc:1034] running on GCE node
I20260808 06:07:06.132563 21161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:07:06.133280 21161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:07:06.134439 21161 hybrid_clock.cc:648] HybridClock initialized: now 1786169226134378 us; error 81 us; skew 500 ppm
Aug 08 06:07:06 dist-test-slave-zj8q krb5kdc[20271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169226, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:07:06.137298 21161 init.cc:377] Logged in from keytab as kudu/127.1.151.129@KRBTEST.COM (short username kudu)
I20260808 06:07:06.138113 21161 webserver.cc:460] Webserver started at http://127.1.151.129:35205/ using document root <none> and password file <none>
I20260808 06:07:06.138670 21161 fs_manager.cc:359] Metadata directory not provided
I20260808 06:07:06.138738 21161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:07:06.138957 21161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 06:07:06 dist-test-slave-zj8q krb5kdc[20271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169226, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for HTTP/127.1.151.148@KRBTEST.COM
I20260808 06:07:06.158910 21161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-0/data/instance:
uuid: "a9584f1b8a1f4b13abab271fa94b606e"
format_stamp: "Formatted at 2026-08-08 06:07:06 on dist-test-slave-zj8q"
server_key: "e8b32811cacaf31941736ac5bbd7c757"
server_key_iv: "9042454f4c4e11de8a4ab2e84a72e66a"
server_key_version: "kuduclusterkey@0"
I20260808 06:07:06.159577 21161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance:
uuid: "a9584f1b8a1f4b13abab271fa94b606e"
format_stamp: "Formatted at 2026-08-08 06:07:06 on dist-test-slave-zj8q"
server_key: "e8b32811cacaf31941736ac5bbd7c757"
server_key_iv: "9042454f4c4e11de8a4ab2e84a72e66a"
server_key_version: "kuduclusterkey@0"
I20260808 06:07:06.163846 21161 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 06:07:06.182540 21191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:06.184311 21161 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.000s
I20260808 06:07:06.184442 21161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-0/wal
uuid: "a9584f1b8a1f4b13abab271fa94b606e"
format_stamp: "Formatted at 2026-08-08 06:07:06 on dist-test-slave-zj8q"
server_key: "e8b32811cacaf31941736ac5bbd7c757"
server_key_iv: "9042454f4c4e11de8a4ab2e84a72e66a"
server_key_version: "kuduclusterkey@0"
I20260808 06:07:06.184571 21161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:07:06.209049 21161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:07:06.212417 21161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:07:06.212611 21161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:07:06.213210 21161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:07:06.214255 21161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:07:06.214359 21161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:06.214488 21161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:07:06.214570 21161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:06.223968 21161 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.129:45149
I20260808 06:07:06.225150 21161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
I20260808 06:07:06.228379 21363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.129:45149 every 8 connection(s)
I20260808 06:07:06.234987 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 21161
I20260808 06:07:06.235106 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance
I20260808 06:07:06.235431 1630 external_mini_cluster.cc:1428] Setting key c299023be0e0d9336b5940ef91fded7d
Aug 08 06:07:06 dist-test-slave-zj8q krb5kdc[20271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169226, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:07:06.242182 21367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:06.229208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:41407 (local address 127.1.151.190:44349)
0808 06:07:06.229486 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:06.229494 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:06.229821 (+ 327us) server_negotiation.cc:407] Connection header received
0808 06:07:06.230949 (+ 1128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:06.230952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:06.231005 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:06.231097 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:06.232529 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:06.233388 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:06.234720 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:06.235031 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:06.235144 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:06.238119 (+ 2975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:06.238123 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:06.238126 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:06.238173 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:06.239832 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:06.240420 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:06.240423 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:06.240425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:06.240515 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:06.240905 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:06.240908 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:06.240910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:06.241122 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:06.241192 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:06.241873 (+ 681us) server_negotiation.cc:299] Negotiation successful
0808 06:07:06.241994 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":91,"threads_started":1}
I20260808 06:07:06.243729 21366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:06.228310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:44349 (local address 127.1.151.129:41407)
0808 06:07:06.229657 (+ 1347us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:06.229714 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:06.230752 (+ 1038us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:06.231391 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:06.231400 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:06.231742 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:07:06.232368 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:06.232387 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:06.233508 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:06.233510 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:06.234570 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:06.234580 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:06.235328 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:06.235332 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:06.235525 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:06.236282 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:07:06.236302 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:07:06.237924 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:07:06.239941 (+ 2017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:06.239948 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:06.239966 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:06.240305 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:06.240633 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:06.240636 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:06.240638 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:06.240803 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:06.241280 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:07:06.241286 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:07:06.241590 (+ 304us) client_negotiation.cc:769] Sending connection context
0808 06:07:06.242393 (+ 803us) client_negotiation.cc:240] Negotiation successful
0808 06:07:06.242719 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1116,"thread_start_us":149,"threads_started":1}
I20260808 06:07:06.245168 21364 heartbeater.cc:347] Connected to a master server at 127.1.151.190:44349
I20260808 06:07:06.245456 21364 heartbeater.cc:464] Registering TS with master...
I20260808 06:07:06.246057 21364 heartbeater.cc:511] Master 127.1.151.190:44349 requested a full tablet report, sending...
I20260808 06:07:06.247879 21035 ts_manager.cc:194] Registered new tserver with Master: a9584f1b8a1f4b13abab271fa94b606e (127.1.151.129:45149)
WARNING: no policy specified for kudu/127.1.151.130@KRBTEST.COM; defaulting to no policy
I20260808 06:07:06.249392 21035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.129@KRBTEST.COM'} at 127.1.151.129:41407
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab.
Entry for principal kudu/127.1.151.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab.
I20260808 06:07:06.267135 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.151.148:48363/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.130:0
--local_ip_for_outbound_sockets=127.1.151.130
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:44349
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:40343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 06:07:06 dist-test-slave-zj8q krb5kdc[20271](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786169226, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 06:07:06.427767 21374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:07:06.428117 21374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:07:06.428179 21374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:07:06.434170 21374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:07:06.434245 21374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:07:06.434329 21374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.130
I20260808 06:07:06.442088 21374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:40343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.151.148:48363/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:44349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.130
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:07:06.443406 21374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:07:06.444785 21374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:07:06.457273 21381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:06.457402 21386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:06.457283 21382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:06.458930 21374 server_base.cc:1034] running on GCE node
I20260808 06:07:06.459419 21374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:07:06.460269 21374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:07:06.461478 21374 hybrid_clock.cc:648] HybridClock initialized: now 1786169226461454 us; error 36 us; skew 500 ppm
Aug 08 06:07:06 dist-test-slave-zj8q krb5kdc[20271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169226, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:07:06.465227 21374 init.cc:377] Logged in from keytab as kudu/127.1.151.130@KRBTEST.COM (short username kudu)
I20260808 06:07:06.466320 21374 webserver.cc:460] Webserver started at http://127.1.151.130:38639/ using document root <none> and password file <none>
I20260808 06:07:06.467021 21374 fs_manager.cc:359] Metadata directory not provided
I20260808 06:07:06.467104 21374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:07:06.467321 21374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 06:07:06 dist-test-slave-zj8q krb5kdc[20271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169226, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for HTTP/127.1.151.148@KRBTEST.COM
I20260808 06:07:06.487185 21374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-1/data/instance:
uuid: "0b8cf2d52d5e4ccc86231319427348d5"
format_stamp: "Formatted at 2026-08-08 06:07:06 on dist-test-slave-zj8q"
server_key: "96b74c6116ecb8b26f76ee22575884e0"
server_key_iv: "8721536ad2582a21c1eaa7b75cead697"
server_key_version: "kuduclusterkey@0"
I20260808 06:07:06.487963 21374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance:
uuid: "0b8cf2d52d5e4ccc86231319427348d5"
format_stamp: "Formatted at 2026-08-08 06:07:06 on dist-test-slave-zj8q"
server_key: "96b74c6116ecb8b26f76ee22575884e0"
server_key_iv: "8721536ad2582a21c1eaa7b75cead697"
server_key_version: "kuduclusterkey@0"
I20260808 06:07:06.495750 21374 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260808 06:07:06.514523 21395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:06.516285 21374 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260808 06:07:06.516492 21374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-1/wal
uuid: "0b8cf2d52d5e4ccc86231319427348d5"
format_stamp: "Formatted at 2026-08-08 06:07:06 on dist-test-slave-zj8q"
server_key: "96b74c6116ecb8b26f76ee22575884e0"
server_key_iv: "8721536ad2582a21c1eaa7b75cead697"
server_key_version: "kuduclusterkey@0"
I20260808 06:07:06.516719 21374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:07:06.536581 21374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:07:06.539808 21374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:07:06.539994 21374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:07:06.540675 21374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:07:06.541704 21374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:07:06.541774 21374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:06.541822 21374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:07:06.541848 21374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:06.552531 21374 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.130:35395
I20260808 06:07:06.552577 21573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.130:35395 every 8 connection(s)
I20260808 06:07:06.553797 21374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
I20260808 06:07:06.558079 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 21374
I20260808 06:07:06.558252 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance
I20260808 06:07:06.558565 1630 external_mini_cluster.cc:1428] Setting key bc9d664b3cc69298455cc4087d72aeca
WARNING: no policy specified for kudu/127.1.151.131@KRBTEST.COM; defaulting to no policy
Aug 08 06:07:06 dist-test-slave-zj8q krb5kdc[20271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169226, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.131@KRBTEST.COM" created.
I20260808 06:07:06.589977 21578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:06.556281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:44349 (local address 127.1.151.130:49005)
0808 06:07:06.556780 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:06.556833 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:06.557659 (+ 826us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:06.558283 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:06.558292 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:06.558637 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:07:06.559237 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:06.559257 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:06.560284 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:06.560287 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:06.574570 (+ 14283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:06.574596 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:06.575796 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:06.575801 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:06.576249 (+ 448us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:06.577318 (+ 1069us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:07:06.577344 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:07:06.583446 (+ 6102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:07:06.586221 (+ 2775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:06.586229 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:06.586249 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:06.586671 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:06.587015 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:06.587018 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:06.587020 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:06.587304 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:06.587895 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:07:06.587903 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:07:06.588278 (+ 375us) client_negotiation.cc:769] Sending connection context
0808 06:07:06.588501 (+ 223us) client_negotiation.cc:240] Negotiation successful
0808 06:07:06.588879 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":123,"threads_started":1}
I20260808 06:07:06.591080 21367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:06.555928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:49005 (local address 127.1.151.190:44349)
0808 06:07:06.556099 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:06.556104 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:06.556952 (+ 848us) server_negotiation.cc:407] Connection header received
0808 06:07:06.557871 (+ 919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:06.557875 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:06.557936 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:06.558046 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:06.559403 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:06.560149 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:06.575041 (+ 14892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:06.575407 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:06.575513 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:06.583738 (+ 8225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:06.583744 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:06.583747 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:06.583782 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:06.586043 (+ 2261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:06.586804 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:06.586808 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:06.586809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:06.586908 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:06.587452 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:06.587456 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:06.587458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:06.587718 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:06.587824 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:06.590785 (+ 2961us) server_negotiation.cc:299] Negotiation successful
0808 06:07:06.590925 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 06:07:06.592221 21575 heartbeater.cc:347] Connected to a master server at 127.1.151.190:44349
I20260808 06:07:06.592509 21575 heartbeater.cc:464] Registering TS with master...
I20260808 06:07:06.593096 21575 heartbeater.cc:511] Master 127.1.151.190:44349 requested a full tablet report, sending...
I20260808 06:07:06.594413 21035 ts_manager.cc:194] Registered new tserver with Master: 0b8cf2d52d5e4ccc86231319427348d5 (127.1.151.130:35395)
I20260808 06:07:06.595172 21035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.130@KRBTEST.COM'} at 127.1.151.130:49005
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab.
Entry for principal kudu/127.1.151.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab.
I20260808 06:07:06.614729 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.151.148:48363/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.131:0
--local_ip_for_outbound_sockets=127.1.151.131
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:44349
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:40343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 06:07:06 dist-test-slave-zj8q krb5kdc[20271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169226, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for HTTP/127.1.151.148@KRBTEST.COM
W20260808 06:07:06.778007 21581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:07:06.778451 21581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:07:06.778566 21581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:07:06.784487 21581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:07:06.784569 21581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:07:06.784650 21581 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.131
I20260808 06:07:06.792207 21581 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:40343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.151.148:48363/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:44349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.131
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:07:06.793480 21581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:07:06.794796 21581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:07:06.807358 21591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:06.807466 21592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:06.807351 21595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:06.808511 21581 server_base.cc:1034] running on GCE node
I20260808 06:07:06.808948 21581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:07:06.809721 21581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:07:06.810921 21581 hybrid_clock.cc:648] HybridClock initialized: now 1786169226810894 us; error 37 us; skew 500 ppm
Aug 08 06:07:06 dist-test-slave-zj8q krb5kdc[20271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169226, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:07:06.814481 21581 init.cc:377] Logged in from keytab as kudu/127.1.151.131@KRBTEST.COM (short username kudu)
I20260808 06:07:06.815501 21581 webserver.cc:460] Webserver started at http://127.1.151.131:44769/ using document root <none> and password file <none>
I20260808 06:07:06.816181 21581 fs_manager.cc:359] Metadata directory not provided
I20260808 06:07:06.816259 21581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:07:06.816462 21581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 06:07:06 dist-test-slave-zj8q krb5kdc[20271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169226, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for HTTP/127.1.151.148@KRBTEST.COM
I20260808 06:07:06.831245 21581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-2/data/instance:
uuid: "c0734e0cf1de44bea1447f3fd21084f4"
format_stamp: "Formatted at 2026-08-08 06:07:06 on dist-test-slave-zj8q"
server_key: "1ef5a99bf0aebb79856a0712ea854a96"
server_key_iv: "a88414b04e0f96be08cc79fcf3047d5d"
server_key_version: "kuduclusterkey@0"
I20260808 06:07:06.831854 21581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance:
uuid: "c0734e0cf1de44bea1447f3fd21084f4"
format_stamp: "Formatted at 2026-08-08 06:07:06 on dist-test-slave-zj8q"
server_key: "1ef5a99bf0aebb79856a0712ea854a96"
server_key_iv: "a88414b04e0f96be08cc79fcf3047d5d"
server_key_version: "kuduclusterkey@0"
I20260808 06:07:06.835621 21581 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260808 06:07:06.856707 21603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:06.858474 21581 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.002s sys 0.000s
I20260808 06:07:06.858614 21581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-2/wal
uuid: "c0734e0cf1de44bea1447f3fd21084f4"
format_stamp: "Formatted at 2026-08-08 06:07:06 on dist-test-slave-zj8q"
server_key: "1ef5a99bf0aebb79856a0712ea854a96"
server_key_iv: "a88414b04e0f96be08cc79fcf3047d5d"
server_key_version: "kuduclusterkey@0"
I20260808 06:07:06.858767 21581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:07:06.891081 21581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:07:06.894142 21581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:07:06.894364 21581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:07:06.895022 21581 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:07:06.895918 21581 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:07:06.895968 21581 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:06.896021 21581 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:07:06.896051 21581 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:06.906107 21581 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.131:35871
I20260808 06:07:06.907083 21581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
I20260808 06:07:06.908016 21779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.131:35871 every 8 connection(s)
I20260808 06:07:06.915308 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 21581
I20260808 06:07:06.915447 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance
I20260808 06:07:06.915759 1630 external_mini_cluster.cc:1428] Setting key 34df83b1da849153af402d38c0af60bc
Aug 08 06:07:06 dist-test-slave-zj8q krb5kdc[20271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169226, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:07:06.920634 21367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:06.909032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:51777 (local address 127.1.151.190:44349)
0808 06:07:06.909160 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:06.909165 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:06.910093 (+ 928us) server_negotiation.cc:407] Connection header received
0808 06:07:06.911245 (+ 1152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:06.911249 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:06.911307 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:06.911382 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:06.912454 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:06.912989 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:06.914083 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:06.914416 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:06.914516 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:06.917168 (+ 2652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:06.917171 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:06.917174 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:06.917199 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:06.918779 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:06.919227 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:06.919230 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:06.919232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:06.919313 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:06.919628 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:06.919632 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:06.919634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:06.919814 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:06.919898 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:06.920365 (+ 467us) server_negotiation.cc:299] Negotiation successful
0808 06:07:06.920479 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 06:07:06.921602 21783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:06.909435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:44349 (local address 127.1.151.131:51777)
0808 06:07:06.909940 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:06.909991 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:06.911027 (+ 1036us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:06.911559 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:06.911567 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:06.911847 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:07:06.912311 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:06.912324 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:06.913105 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:06.913108 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:06.913963 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:06.913973 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:06.914664 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:06.914667 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:06.914827 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:06.915393 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:07:06.915415 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:07:06.916999 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:07:06.918873 (+ 1874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:06.918879 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:06.918892 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:06.919133 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:06.919392 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:06.919395 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:06.919397 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:06.919540 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:06.919911 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:07:06.919918 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:07:06.920135 (+ 217us) client_negotiation.cc:769] Sending connection context
0808 06:07:06.920304 (+ 169us) client_negotiation.cc:240] Negotiation successful
0808 06:07:06.920581 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":121,"threads_started":1}
I20260808 06:07:06.922605 21780 heartbeater.cc:347] Connected to a master server at 127.1.151.190:44349
I20260808 06:07:06.922830 21780 heartbeater.cc:464] Registering TS with master...
I20260808 06:07:06.923293 21780 heartbeater.cc:511] Master 127.1.151.190:44349 requested a full tablet report, sending...
I20260808 06:07:06.924304 21035 ts_manager.cc:194] Registered new tserver with Master: c0734e0cf1de44bea1447f3fd21084f4 (127.1.151.131:35871)
I20260808 06:07:06.924836 21035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.131@KRBTEST.COM'} at 127.1.151.131:51777
I20260808 06:07:06.929988 1630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 06:07:06.930790 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 21161
I20260808 06:07:06.958323 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 21374
I20260808 06:07:06.977617 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 21581
I20260808 06:07:06.992224 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 20980
I20260808 06:07:07.009029 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:44349
--webserver_interface=127.1.151.190
--webserver_port=43999
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:40343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:44349
--ranger_config_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.151.148:48363/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.151.148:48363/kms with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:07:07.118700 21784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:07:07.118945 21784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:07:07.118983 21784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:07:07.123076 21784 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:07:07.123129 21784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:07:07.123148 21784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:07:07.123162 21784 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:07:07.123178 21784 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:07:07.128430 21784 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:40343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.151.148:48363/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:44349
--ranger_config_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:44349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=43999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:07:07.129367 21784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:07:07.130360 21784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:07:07.138429 21793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:07.138505 21784 server_base.cc:1034] running on GCE node
W20260808 06:07:07.138507 21794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:07.138420 21797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:07.139267 21784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:07:07.140260 21784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:07:07.141427 21784 hybrid_clock.cc:648] HybridClock initialized: now 1786169227141412 us; error 28 us; skew 500 ppm
Aug 08 06:07:07 dist-test-slave-zj8q krb5kdc[20271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169227, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:07:07.144088 21784 init.cc:377] Logged in from keytab as kudu/127.1.151.190@KRBTEST.COM (short username kudu)
I20260808 06:07:07.144850 21784 webserver.cc:460] Webserver started at http://127.1.151.190:43999/ using document root <none> and password file <none>
I20260808 06:07:07.145323 21784 fs_manager.cc:359] Metadata directory not provided
I20260808 06:07:07.145387 21784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 06:07:07 dist-test-slave-zj8q krb5kdc[20271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169227, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for HTTP/127.1.151.148@KRBTEST.COM
I20260808 06:07:07.182732 21784 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 06:07:07.185055 21810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:07.186362 21784 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 06:07:07.186484 21784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "3eb419af59bc4da6a8074f1dc3169721"
format_stamp: "Formatted at 2026-08-08 06:07:05 on dist-test-slave-zj8q"
server_key: "90fbab75ac518939fa64f5e6ecf999ea"
server_key_iv: "117311e7459401b7a2ecec7159fcc7e1"
server_key_version: "kuduclusterkey@0"
I20260808 06:07:07.187009 21784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:07:07.206774 21784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:07:07.209969 21784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:07:07.210129 21784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:07:07.217121 21784 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:44349
I20260808 06:07:07.217144 21904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:44349 every 8 connection(s)
I20260808 06:07:07.218128 21784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:07:07.222435 21905 sys_catalog.cc:263] Verifying existing consensus state
I20260808 06:07:07.224051 21905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721: Bootstrap starting.
I20260808 06:07:07.226043 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 21784
I20260808 06:07:07.226465 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.151.148:48363/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.129:45149
--local_ip_for_outbound_sockets=127.1.151.129
--tserver_master_addrs=127.1.151.190:44349
--webserver_port=35205
--webserver_interface=127.1.151.129
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:40343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.151.148:48363/kms with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 06:07:07.240125 21905 log.cc:824] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721: Log is configured to *not* fsync() on all Append() calls
I20260808 06:07:07.246831 21905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 06:07:07.247174 21905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721: Bootstrap complete.
I20260808 06:07:07.251171 21905 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eb419af59bc4da6a8074f1dc3169721" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44349 } }
I20260808 06:07:07.251559 21905 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb419af59bc4da6a8074f1dc3169721, State: Initialized, Role: FOLLOWER
I20260808 06:07:07.252050 21905 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eb419af59bc4da6a8074f1dc3169721" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44349 } }
I20260808 06:07:07.252178 21905 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:07:07.252244 21905 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:07:07.252338 21905 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721 [term 1 FOLLOWER]: Advancing to term 2
I20260808 06:07:07.254177 21905 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eb419af59bc4da6a8074f1dc3169721" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44349 } }
I20260808 06:07:07.254501 21905 leader_election.cc:302] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3eb419af59bc4da6a8074f1dc3169721; no voters:
I20260808 06:07:07.254793 21905 leader_election.cc:288] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 06:07:07.254891 21912 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 06:07:07.255064 21912 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721 [term 2 LEADER]: Becoming Leader. State: Replica: 3eb419af59bc4da6a8074f1dc3169721, State: Running, Role: LEADER
I20260808 06:07:07.255360 21912 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eb419af59bc4da6a8074f1dc3169721" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44349 } }
I20260808 06:07:07.256017 21905 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:07:07.256721 21914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3eb419af59bc4da6a8074f1dc3169721" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eb419af59bc4da6a8074f1dc3169721" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44349 } } }
I20260808 06:07:07.256857 21914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721 [sys.catalog]: This master's current role is: LEADER
I20260808 06:07:07.257436 21915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3eb419af59bc4da6a8074f1dc3169721. Latest consensus state: current_term: 2 leader_uuid: "3eb419af59bc4da6a8074f1dc3169721" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eb419af59bc4da6a8074f1dc3169721" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44349 } } }
I20260808 06:07:07.257602 21915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721 [sys.catalog]: This master's current role is: LEADER
I20260808 06:07:07.257810 21925 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:07:07.258286 21905 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 06:07:07.264734 21925 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:07:07.265448 21925 catalog_manager.cc:1259] Loaded cluster ID: 522cec458e1d4d32ac67edfd42e16e9c
I20260808 06:07:07.265511 21925 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:07:07.268132 21925 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:07:07.268857 21925 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721: Loaded TSK: 0
I20260808 06:07:07.270237 21925 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 06:07:07.368266 21908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:07:07.368577 21908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:07:07.368654 21908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:07:07.374498 21908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:07:07.374569 21908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:07:07.374722 21908 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.129
I20260808 06:07:07.382009 21908 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:40343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.151.148:48363/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.129:45149
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.151.129
--webserver_port=35205
--tserver_master_addrs=127.1.151.190:44349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.129
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:07:07.383289 21908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:07:07.384568 21908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:07:07.397006 21964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:07.397073 21963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:07.397330 21908 server_base.cc:1034] running on GCE node
W20260808 06:07:07.397261 21968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:07.397993 21908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:07:07.398785 21908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:07:07.399984 21908 hybrid_clock.cc:648] HybridClock initialized: now 1786169227399961 us; error 37 us; skew 500 ppm
Aug 08 06:07:07 dist-test-slave-zj8q krb5kdc[20271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169227, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:07:07.403460 21908 init.cc:377] Logged in from keytab as kudu/127.1.151.129@KRBTEST.COM (short username kudu)
I20260808 06:07:07.404554 21908 webserver.cc:460] Webserver started at http://127.1.151.129:35205/ using document root <none> and password file <none>
I20260808 06:07:07.405239 21908 fs_manager.cc:359] Metadata directory not provided
I20260808 06:07:07.405349 21908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 06:07:07 dist-test-slave-zj8q krb5kdc[20271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169227, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for HTTP/127.1.151.148@KRBTEST.COM
I20260808 06:07:07.427233 21908 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 06:07:07.434850 21980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:07.439669 21908 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.002s sys 0.000s
I20260808 06:07:07.439988 21908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-0/wal
uuid: "a9584f1b8a1f4b13abab271fa94b606e"
format_stamp: "Formatted at 2026-08-08 06:07:06 on dist-test-slave-zj8q"
server_key: "e8b32811cacaf31941736ac5bbd7c757"
server_key_iv: "9042454f4c4e11de8a4ab2e84a72e66a"
server_key_version: "kuduclusterkey@0"
I20260808 06:07:07.440762 21908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:07:07.460865 21908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:07:07.464447 21908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:07:07.464635 21908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:07:07.465240 21908 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:07:07.466199 21908 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:07:07.466256 21908 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:07.466311 21908 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:07:07.466336 21908 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:07.476847 21908 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.129:45149
I20260808 06:07:07.476902 22147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.129:45149 every 8 connection(s)
I20260808 06:07:07.478039 21908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
I20260808 06:07:07.486243 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 21908
I20260808 06:07:07.486716 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.151.148:48363/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.130:35395
--local_ip_for_outbound_sockets=127.1.151.130
--tserver_master_addrs=127.1.151.190:44349
--webserver_port=38639
--webserver_interface=127.1.151.130
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:40343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.151.148:48363/kms with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 06:07:07 dist-test-slave-zj8q krb5kdc[20271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169227, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:07:07.498543 22152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:07.480982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:44349 (local address 127.1.151.129:38559)
0808 06:07:07.481751 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:07.481807 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:07.483498 (+ 1691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:07.485193 (+ 1695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:07.485203 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:07.485612 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:07:07.486289 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:07.486311 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:07.488214 (+ 1903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:07.488218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:07.489569 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:07.489581 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:07.490727 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:07.490731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:07.490901 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:07.491488 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:07:07.491504 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:07:07.492988 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:07:07.495220 (+ 2232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:07.495226 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:07.495240 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:07.495572 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:07.496006 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:07.496009 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:07.496011 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:07.496188 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:07.496952 (+ 764us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:07:07.496959 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:07:07.497245 (+ 286us) client_negotiation.cc:769] Sending connection context
0808 06:07:07.497383 (+ 138us) client_negotiation.cc:240] Negotiation successful
0808 06:07:07.497632 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":149,"threads_started":1}
I20260808 06:07:07.500622 22153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:07.482508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:38559 (local address 127.1.151.190:44349)
0808 06:07:07.483987 (+ 1479us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:07.484007 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:07.484049 (+ 42us) server_negotiation.cc:407] Connection header received
0808 06:07:07.484344 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:07.484359 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:07.484737 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:07.485084 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:07.486502 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:07.488052 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:07.489739 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:07.490516 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:07.490727 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:07.493173 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:07.493179 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:07.493195 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:07.493228 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:07.495078 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:07.495739 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:07.495745 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:07.495751 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:07.495890 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:07.496314 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:07.496317 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:07.496319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:07.496811 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:07.499269 (+ 2458us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:07.499415 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 06:07:07.499711 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1318,"thread_start_us":108,"threads_started":1}
I20260808 06:07:07.503153 22148 heartbeater.cc:347] Connected to a master server at 127.1.151.190:44349
I20260808 06:07:07.503410 22148 heartbeater.cc:464] Registering TS with master...
I20260808 06:07:07.503985 22148 heartbeater.cc:511] Master 127.1.151.190:44349 requested a full tablet report, sending...
W20260808 06:07:07.506043 22148 heartbeater.cc:644] Failed to heartbeat to 127.1.151.190:44349 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 06:07:07.506206 22148 heartbeater.cc:464] Registering TS with master...
I20260808 06:07:07.506382 22148 heartbeater.cc:511] Master 127.1.151.190:44349 requested a full tablet report, sending...
I20260808 06:07:07.507200 22148 heartbeater.cc:464] Registering TS with master...
I20260808 06:07:07.507367 22148 heartbeater.cc:511] Master 127.1.151.190:44349 requested a full tablet report, sending...
W20260808 06:07:07.508070 22148 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 06:07:07.622689 22160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:07:07.622928 22160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:07:07.622983 22160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:07:07.627074 22160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:07:07.627135 22160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:07:07.627209 22160 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.130
I20260808 06:07:07.632313 22160 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:40343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.151.148:48363/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.130:35395
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.151.130
--webserver_port=38639
--tserver_master_addrs=127.1.151.190:44349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22160
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.130
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:07:07.633311 22160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:07:07.634356 22160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:07:07.645048 22166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:07.645592 22169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:07.648479 22167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:07.648861 22160 server_base.cc:1034] running on GCE node
I20260808 06:07:07.649344 22160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:07:07.649972 22160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:07:07.651151 22160 hybrid_clock.cc:648] HybridClock initialized: now 1786169227651129 us; error 39 us; skew 500 ppm
Aug 08 06:07:07 dist-test-slave-zj8q krb5kdc[20271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169227, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:07:07.654862 22160 init.cc:377] Logged in from keytab as kudu/127.1.151.130@KRBTEST.COM (short username kudu)
I20260808 06:07:07.655896 22160 webserver.cc:460] Webserver started at http://127.1.151.130:38639/ using document root <none> and password file <none>
I20260808 06:07:07.656576 22160 fs_manager.cc:359] Metadata directory not provided
I20260808 06:07:07.656731 22160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 06:07:07 dist-test-slave-zj8q krb5kdc[20271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169227, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for HTTP/127.1.151.148@KRBTEST.COM
I20260808 06:07:07.681463 22160 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 06:07:07.687651 22183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:07.689206 22160 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260808 06:07:07.689342 22160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-1/wal
uuid: "0b8cf2d52d5e4ccc86231319427348d5"
format_stamp: "Formatted at 2026-08-08 06:07:06 on dist-test-slave-zj8q"
server_key: "96b74c6116ecb8b26f76ee22575884e0"
server_key_iv: "8721536ad2582a21c1eaa7b75cead697"
server_key_version: "kuduclusterkey@0"
I20260808 06:07:07.690047 22160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:07:07.704650 22160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:07:07.707991 22160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:07:07.708186 22160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:07:07.708869 22160 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:07:07.709888 22160 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:07:07.709952 22160 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:07.710016 22160 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:07:07.710047 22160 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:07.721029 22160 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.130:35395
I20260808 06:07:07.722303 22160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
I20260808 06:07:07.722854 22362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.130:35395 every 8 connection(s)
I20260808 06:07:07.730006 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 22160
I20260808 06:07:07.730535 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.151.148:48363/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.131:35871
--local_ip_for_outbound_sockets=127.1.151.131
--tserver_master_addrs=127.1.151.190:44349
--webserver_port=44769
--webserver_interface=127.1.151.131
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:40343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.151.148:48363/kms with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 06:07:07 dist-test-slave-zj8q krb5kdc[20271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169227, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:07:07.743072 22153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:07.724910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:34935 (local address 127.1.151.190:44349)
0808 06:07:07.725103 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:07.725108 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:07.729858 (+ 4750us) server_negotiation.cc:407] Connection header received
0808 06:07:07.731165 (+ 1307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:07.731170 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:07.731233 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:07.731295 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:07.732990 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:07.733843 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:07.735634 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:07.735994 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:07.736084 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:07.739134 (+ 3050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:07.739139 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:07.739142 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:07.739173 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:07.740745 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:07.741306 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:07.741310 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:07.741311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:07.741403 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:07.741786 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:07.741789 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:07.741791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:07.742000 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:07.742076 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:07.742769 (+ 693us) server_negotiation.cc:299] Negotiation successful
0808 06:07:07.742905 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 06:07:07.744796 22367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:07.725404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:44349 (local address 127.1.151.130:34935)
0808 06:07:07.729696 (+ 4292us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:07.729751 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:07.730901 (+ 1150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:07.731733 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:07.731742 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:07.732092 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:07:07.732796 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:07.732819 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:07.734013 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:07.734016 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:07.735471 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:07.735483 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:07.736310 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:07.736314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:07.736520 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:07.737160 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:07:07.737182 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:07:07.738944 (+ 1762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:07:07.740880 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:07.740886 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:07.740901 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:07.741210 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:07.741501 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:07.741504 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:07.741505 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:07.741681 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:07.742143 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:07:07.742172 (+ 29us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:07:07.742484 (+ 312us) client_negotiation.cc:769] Sending connection context
0808 06:07:07.743358 (+ 874us) client_negotiation.cc:240] Negotiation successful
0808 06:07:07.743719 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4046,"thread_start_us":143,"threads_started":1}
I20260808 06:07:07.746248 22363 heartbeater.cc:347] Connected to a master server at 127.1.151.190:44349
I20260808 06:07:07.746513 22363 heartbeater.cc:464] Registering TS with master...
I20260808 06:07:07.747138 22363 heartbeater.cc:511] Master 127.1.151.190:44349 requested a full tablet report, sending...
W20260808 06:07:07.752183 22363 heartbeater.cc:644] Failed to heartbeat to 127.1.151.190:44349 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 06:07:07.752322 22363 heartbeater.cc:464] Registering TS with master...
I20260808 06:07:07.752511 22363 heartbeater.cc:511] Master 127.1.151.190:44349 requested a full tablet report, sending...
I20260808 06:07:07.753345 22363 heartbeater.cc:464] Registering TS with master...
I20260808 06:07:07.753518 22363 heartbeater.cc:511] Master 127.1.151.190:44349 requested a full tablet report, sending...
W20260808 06:07:07.754274 22363 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 06:07:07.877501 22369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:07:07.877816 22369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:07:07.877877 22369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:07:07.883817 22369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:07:07.883890 22369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:07:07.883971 22369 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.131
I20260808 06:07:07.891605 22369 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:40343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.151.148:48363/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.131:35871
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.151.131
--webserver_port=44769
--tserver_master_addrs=127.1.151.190:44349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.131
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:07:07.892839 22369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:07:07.894124 22369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:07:07.905869 22380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:07.906021 22375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:07.906286 22369 server_base.cc:1034] running on GCE node
W20260808 06:07:07.910297 22377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:07.910737 22369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:07:07.911505 22369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:07:07.912690 22369 hybrid_clock.cc:648] HybridClock initialized: now 1786169227912667 us; error 37 us; skew 500 ppm
Aug 08 06:07:07 dist-test-slave-zj8q krb5kdc[20271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169227, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:07:07.915834 22369 init.cc:377] Logged in from keytab as kudu/127.1.151.131@KRBTEST.COM (short username kudu)
I20260808 06:07:07.916759 22369 webserver.cc:460] Webserver started at http://127.1.151.131:44769/ using document root <none> and password file <none>
I20260808 06:07:07.917326 22369 fs_manager.cc:359] Metadata directory not provided
I20260808 06:07:07.917409 22369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 06:07:07 dist-test-slave-zj8q krb5kdc[20271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169227, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for HTTP/127.1.151.148@KRBTEST.COM
I20260808 06:07:07.938076 22369 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.003s sys 0.000s
I20260808 06:07:07.955107 22393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:07.956960 22369 fs_manager.cc:725] Time spent opening block manager: real 0.017s user 0.003s sys 0.000s
I20260808 06:07:07.957129 22369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-2/wal
uuid: "c0734e0cf1de44bea1447f3fd21084f4"
format_stamp: "Formatted at 2026-08-08 06:07:06 on dist-test-slave-zj8q"
server_key: "1ef5a99bf0aebb79856a0712ea854a96"
server_key_iv: "a88414b04e0f96be08cc79fcf3047d5d"
server_key_version: "kuduclusterkey@0"
I20260808 06:07:07.957911 22369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:07:07.978790 22369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:07:07.982124 22369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:07:07.982355 22369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:07:07.983090 22369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:07:07.984162 22369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:07:07.984230 22369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:07.984287 22369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:07:07.984319 22369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:07.995386 22369 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.131:35871
I20260808 06:07:07.995656 22611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.131:35871 every 8 connection(s)
I20260808 06:07:07.996436 22369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
I20260808 06:07:07.998811 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 22369
Aug 08 06:07:08 dist-test-slave-zj8q krb5kdc[20271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169227, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:07:08.009394 22619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:07.999539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60314 (local address 127.1.151.190:44349)
0808 06:07:08.000217 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:08.000227 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:08.000241 (+ 14us) server_negotiation.cc:407] Connection header received
0808 06:07:08.000287 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:08.000290 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:08.000338 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:08.000435 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:08.001611 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:08.002427 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:08.003614 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:08.003979 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:08.004072 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:08.005975 (+ 1903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:08.005979 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:08.005981 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:08.006006 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:08.007420 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:08.008130 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:08.008133 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:08.008135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:08.008258 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:08.008555 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:08.008559 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:08.008561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:08.008745 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:08.008928 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:08.009102 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 06:07:08.009214 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":96,"threads_started":1}
I20260808 06:07:08.011454 22153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:07.998748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:42609 (local address 127.1.151.190:44349)
0808 06:07:07.998877 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:07.998882 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:08.000475 (+ 1593us) server_negotiation.cc:407] Connection header received
0808 06:07:08.001390 (+ 915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:08.001394 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:08.001443 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:08.001513 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:08.002794 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:08.003283 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:08.004587 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:08.004812 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:08.004904 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:08.007628 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:08.007632 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:08.007634 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:08.007658 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:08.009178 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:08.009920 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:08.009923 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:08.009925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:08.010010 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:08.010402 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:08.010405 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:08.010406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:08.010596 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:08.010697 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:08.011268 (+ 571us) server_negotiation.cc:299] Negotiation successful
0808 06:07:08.011354 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 06:07:08.012501 22616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:07.999231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:44349 (local address 127.1.151.131:42609)
0808 06:07:08.000340 (+ 1109us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:08.000396 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:08.001202 (+ 806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:08.001701 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:08.001711 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:08.002032 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:07:08.002668 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:08.002684 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:08.003400 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:08.003404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:08.004482 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:08.004492 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:08.004969 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:08.004973 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:08.005167 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:08.005737 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:07:08.005756 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:07:08.007411 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:07:08.009339 (+ 1928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:08.009344 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:08.009358 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:08.009708 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:08.010106 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:08.010109 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:08.010111 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:08.010314 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:08.010712 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:07:08.010719 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:07:08.011013 (+ 294us) client_negotiation.cc:769] Sending connection context
0808 06:07:08.011216 (+ 203us) client_negotiation.cc:240] Negotiation successful
0808 06:07:08.011527 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":909,"thread_start_us":136,"threads_started":1}
I20260808 06:07:08.015333 22612 heartbeater.cc:347] Connected to a master server at 127.1.151.190:44349
I20260808 06:07:08.015717 22612 heartbeater.cc:464] Registering TS with master...
I20260808 06:07:08.016350 22612 heartbeater.cc:511] Master 127.1.151.190:44349 requested a full tablet report, sending...
I20260808 06:07:08.017897 21839 ts_manager.cc:194] Registered new tserver with Master: c0734e0cf1de44bea1447f3fd21084f4 (127.1.151.131:35871)
I20260808 06:07:08.019251 21839 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.131@KRBTEST.COM'} at 127.1.151.131:42609
Aug 08 06:07:08 dist-test-slave-zj8q krb5kdc[20271](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786169228, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:07:08.508291 22148 heartbeater.cc:464] Registering TS with master...
I20260808 06:07:08.508569 22148 heartbeater.cc:511] Master 127.1.151.190:44349 requested a full tablet report, sending...
I20260808 06:07:08.509322 21839 ts_manager.cc:194] Registered new tserver with Master: a9584f1b8a1f4b13abab271fa94b606e (127.1.151.129:45149)
I20260808 06:07:08.510017 21839 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.129@KRBTEST.COM'} at 127.1.151.129:38559
Aug 08 06:07:08 dist-test-slave-zj8q krb5kdc[20271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169228, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for HTTP/127.1.151.148@KRBTEST.COM
I20260808 06:07:08.754506 22363 heartbeater.cc:464] Registering TS with master...
I20260808 06:07:08.754822 22363 heartbeater.cc:511] Master 127.1.151.190:44349 requested a full tablet report, sending...
I20260808 06:07:08.755591 21839 ts_manager.cc:194] Registered new tserver with Master: 0b8cf2d52d5e4ccc86231319427348d5 (127.1.151.130:35395)
I20260808 06:07:08.756387 21839 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.130@KRBTEST.COM'} at 127.1.151.130:34935
I20260808 06:07:08.759424 1630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 06:07:08.770038 22637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:08.761778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60322 (local address 127.1.151.190:44349)
0808 06:07:08.762305 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:08.762310 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:08.762725 (+ 415us) server_negotiation.cc:407] Connection header received
0808 06:07:08.762834 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:08.762838 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:08.762886 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:08.762963 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:08.763692 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:08.764303 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:08.765209 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:08.765444 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:08.765545 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:08.766865 (+ 1320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:08.766869 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:08.766872 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:08.766898 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:08.768298 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:08.768691 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:08.768695 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:08.768697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:08.768776 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:08.769041 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:08.769044 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:08.769046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:08.769268 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:08.769399 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:08.769741 (+ 342us) server_negotiation.cc:299] Negotiation successful
0808 06:07:08.769859 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":69,"threads_started":1}
I20260808 06:07:08.773885 21839 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60322:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 06:07:08.778296 21839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 06:07:08.796944 22649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:08.789271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:35395 (local address 127.0.0.1:60116)
0808 06:07:08.790574 (+ 1303us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:08.790621 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:08.790752 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:08.791716 (+ 964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:08.791722 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:08.791745 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:07:08.791874 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:08.791880 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:08.793332 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:08.793336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:08.795062 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:08.795074 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:08.796425 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:08.796428 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:08.796546 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:08.796585 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 06:07:08.796685 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 06:07:08.796796 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1130,"thread_start_us":98,"threads_started":1}
I20260808 06:07:08.797541 22647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:08.789385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60116 (local address 127.1.151.130:35395)
0808 06:07:08.791020 (+ 1635us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:08.791027 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:08.791061 (+ 34us) server_negotiation.cc:407] Connection header received
0808 06:07:08.791135 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:08.791141 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:08.791408 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:08.791548 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:07:08.791999 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:08.793204 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:08.795199 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:08.796128 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:08.796224 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:08.796363 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:08.797299 (+ 936us) server_negotiation.cc:299] Negotiation successful
0808 06:07:08.797404 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1528,"thread_start_us":73,"threads_started":1}
I20260808 06:07:08.798825 22646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:08.790036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:45149 (local address 127.0.0.1:56496)
0808 06:07:08.791709 (+ 1673us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:08.791722 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:08.791808 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:08.792465 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:08.792468 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:08.792485 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:07:08.792573 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:08.792579 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:08.794786 (+ 2207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:08.794789 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:08.795956 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:08.795968 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:08.798426 (+ 2458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:08.798430 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:08.798522 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:08.798546 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 06:07:08.798618 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 06:07:08.798690 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1618,"thread_start_us":126,"threads_started":1}
I20260808 06:07:08.799949 22650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:08.790213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56496 (local address 127.1.151.129:45149)
0808 06:07:08.791580 (+ 1367us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:08.791586 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:08.791892 (+ 306us) server_negotiation.cc:407] Connection header received
0808 06:07:08.791970 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:08.791974 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:08.792169 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:08.792304 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:07:08.792697 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:08.794652 (+ 1955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:08.796114 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:08.798078 (+ 1964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:08.798216 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:08.798356 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:08.799714 (+ 1358us) server_negotiation.cc:299] Negotiation successful
0808 06:07:08.799811 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1263,"thread_start_us":73,"threads_started":1}
I20260808 06:07:08.800586 22264 tablet_service.cc:1467] Processing CreateTablet for tablet 721fcb97fdbc429cbeae5828cad858c0 (DEFAULT_TABLE table=test-table [id=76d5e7b4fb0b4f6295f8af2e4ca59c2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:07:08.801733 22264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 721fcb97fdbc429cbeae5828cad858c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:07:08.805058 22648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:08.789999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:35871 (local address 127.0.0.1:59550)
0808 06:07:08.791246 (+ 1247us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:08.791261 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:08.791353 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:08.800602 (+ 9249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:08.800606 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:08.800625 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:07:08.800725 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:08.800730 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:08.802475 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:08.802479 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:08.803584 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:08.803596 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:08.804692 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:08.804696 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:08.804793 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:08.804817 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 06:07:08.804865 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 06:07:08.804925 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1180,"thread_start_us":94,"threads_started":1}
I20260808 06:07:08.805236 22055 tablet_service.cc:1467] Processing CreateTablet for tablet 721fcb97fdbc429cbeae5828cad858c0 (DEFAULT_TABLE table=test-table [id=76d5e7b4fb0b4f6295f8af2e4ca59c2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:07:08.805902 22651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:08.790408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59550 (local address 127.1.151.131:35871)
0808 06:07:08.799277 (+ 8869us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:08.799284 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:08.799307 (+ 23us) server_negotiation.cc:407] Connection header received
0808 06:07:08.799380 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:08.799385 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:08.799557 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:08.800899 (+ 1342us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:07:08.801053 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:08.802333 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:08.803747 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:08.804548 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:08.805397 (+ 849us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:08.805560 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:08.805611 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 06:07:08.805702 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8761,"thread_start_us":86,"threads_started":1}
I20260808 06:07:08.808905 22489 tablet_service.cc:1467] Processing CreateTablet for tablet 721fcb97fdbc429cbeae5828cad858c0 (DEFAULT_TABLE table=test-table [id=76d5e7b4fb0b4f6295f8af2e4ca59c2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:07:08.809866 22489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 721fcb97fdbc429cbeae5828cad858c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:07:08.811264 22055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 721fcb97fdbc429cbeae5828cad858c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:07:08.814395 22652 tablet_bootstrap.cc:492] T 721fcb97fdbc429cbeae5828cad858c0 P 0b8cf2d52d5e4ccc86231319427348d5: Bootstrap starting.
I20260808 06:07:08.816694 22652 tablet_bootstrap.cc:654] T 721fcb97fdbc429cbeae5828cad858c0 P 0b8cf2d52d5e4ccc86231319427348d5: Neither blocks nor log segments found. Creating new log.
I20260808 06:07:08.817560 22652 log.cc:824] T 721fcb97fdbc429cbeae5828cad858c0 P 0b8cf2d52d5e4ccc86231319427348d5: Log is configured to *not* fsync() on all Append() calls
I20260808 06:07:08.824417 22654 tablet_bootstrap.cc:492] T 721fcb97fdbc429cbeae5828cad858c0 P a9584f1b8a1f4b13abab271fa94b606e: Bootstrap starting.
I20260808 06:07:08.826808 22654 tablet_bootstrap.cc:654] T 721fcb97fdbc429cbeae5828cad858c0 P a9584f1b8a1f4b13abab271fa94b606e: Neither blocks nor log segments found. Creating new log.
I20260808 06:07:08.827698 22654 log.cc:824] T 721fcb97fdbc429cbeae5828cad858c0 P a9584f1b8a1f4b13abab271fa94b606e: Log is configured to *not* fsync() on all Append() calls
I20260808 06:07:08.829901 22654 tablet_bootstrap.cc:492] T 721fcb97fdbc429cbeae5828cad858c0 P a9584f1b8a1f4b13abab271fa94b606e: No bootstrap required, opened a new log
I20260808 06:07:08.830205 22654 ts_tablet_manager.cc:1397] T 721fcb97fdbc429cbeae5828cad858c0 P a9584f1b8a1f4b13abab271fa94b606e: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260808 06:07:08.830430 22652 tablet_bootstrap.cc:492] T 721fcb97fdbc429cbeae5828cad858c0 P 0b8cf2d52d5e4ccc86231319427348d5: No bootstrap required, opened a new log
I20260808 06:07:08.830634 22652 ts_tablet_manager.cc:1397] T 721fcb97fdbc429cbeae5828cad858c0 P 0b8cf2d52d5e4ccc86231319427348d5: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.002s
I20260808 06:07:08.834678 22652 raft_consensus.cc:348] T 721fcb97fdbc429cbeae5828cad858c0 P 0b8cf2d52d5e4ccc86231319427348d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b8cf2d52d5e4ccc86231319427348d5" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 35395 } } peers { permanent_uuid: "c0734e0cf1de44bea1447f3fd21084f4" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35871 } } peers { permanent_uuid: "a9584f1b8a1f4b13abab271fa94b606e" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 45149 } }
I20260808 06:07:08.834957 22652 raft_consensus.cc:374] T 721fcb97fdbc429cbeae5828cad858c0 P 0b8cf2d52d5e4ccc86231319427348d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:07:08.835009 22652 raft_consensus.cc:729] T 721fcb97fdbc429cbeae5828cad858c0 P 0b8cf2d52d5e4ccc86231319427348d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b8cf2d52d5e4ccc86231319427348d5, State: Initialized, Role: FOLLOWER
I20260808 06:07:08.835465 22652 consensus_queue.cc:260] T 721fcb97fdbc429cbeae5828cad858c0 P 0b8cf2d52d5e4ccc86231319427348d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b8cf2d52d5e4ccc86231319427348d5" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 35395 } } peers { permanent_uuid: "c0734e0cf1de44bea1447f3fd21084f4" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35871 } } peers { permanent_uuid: "a9584f1b8a1f4b13abab271fa94b606e" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 45149 } }
I20260808 06:07:08.835937 22654 raft_consensus.cc:348] T 721fcb97fdbc429cbeae5828cad858c0 P a9584f1b8a1f4b13abab271fa94b606e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b8cf2d52d5e4ccc86231319427348d5" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 35395 } } peers { permanent_uuid: "c0734e0cf1de44bea1447f3fd21084f4" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35871 } } peers { permanent_uuid: "a9584f1b8a1f4b13abab271fa94b606e" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 45149 } }
I20260808 06:07:08.836246 22654 raft_consensus.cc:374] T 721fcb97fdbc429cbeae5828cad858c0 P a9584f1b8a1f4b13abab271fa94b606e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:07:08.836334 22654 raft_consensus.cc:729] T 721fcb97fdbc429cbeae5828cad858c0 P a9584f1b8a1f4b13abab271fa94b606e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9584f1b8a1f4b13abab271fa94b606e, State: Initialized, Role: FOLLOWER
I20260808 06:07:08.836539 22652 ts_tablet_manager.cc:1428] T 721fcb97fdbc429cbeae5828cad858c0 P 0b8cf2d52d5e4ccc86231319427348d5: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260808 06:07:08.836899 22363 heartbeater.cc:503] Master 127.1.151.190:44349 was elected leader, sending a full tablet report...
I20260808 06:07:08.836860 22654 consensus_queue.cc:260] T 721fcb97fdbc429cbeae5828cad858c0 P a9584f1b8a1f4b13abab271fa94b606e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b8cf2d52d5e4ccc86231319427348d5" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 35395 } } peers { permanent_uuid: "c0734e0cf1de44bea1447f3fd21084f4" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35871 } } peers { permanent_uuid: "a9584f1b8a1f4b13abab271fa94b606e" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 45149 } }
I20260808 06:07:08.837234 22655 tablet_bootstrap.cc:492] T 721fcb97fdbc429cbeae5828cad858c0 P c0734e0cf1de44bea1447f3fd21084f4: Bootstrap starting.
I20260808 06:07:08.839798 22655 tablet_bootstrap.cc:654] T 721fcb97fdbc429cbeae5828cad858c0 P c0734e0cf1de44bea1447f3fd21084f4: Neither blocks nor log segments found. Creating new log.
I20260808 06:07:08.840660 22655 log.cc:824] T 721fcb97fdbc429cbeae5828cad858c0 P c0734e0cf1de44bea1447f3fd21084f4: Log is configured to *not* fsync() on all Append() calls
I20260808 06:07:08.840883 22654 ts_tablet_manager.cc:1428] T 721fcb97fdbc429cbeae5828cad858c0 P a9584f1b8a1f4b13abab271fa94b606e: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20260808 06:07:08.841286 22148 heartbeater.cc:503] Master 127.1.151.190:44349 was elected leader, sending a full tablet report...
I20260808 06:07:08.843266 22655 tablet_bootstrap.cc:492] T 721fcb97fdbc429cbeae5828cad858c0 P c0734e0cf1de44bea1447f3fd21084f4: No bootstrap required, opened a new log
I20260808 06:07:08.843456 22655 ts_tablet_manager.cc:1397] T 721fcb97fdbc429cbeae5828cad858c0 P c0734e0cf1de44bea1447f3fd21084f4: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260808 06:07:08.847992 22655 raft_consensus.cc:348] T 721fcb97fdbc429cbeae5828cad858c0 P c0734e0cf1de44bea1447f3fd21084f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b8cf2d52d5e4ccc86231319427348d5" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 35395 } } peers { permanent_uuid: "c0734e0cf1de44bea1447f3fd21084f4" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35871 } } peers { permanent_uuid: "a9584f1b8a1f4b13abab271fa94b606e" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 45149 } }
I20260808 06:07:08.848268 22655 raft_consensus.cc:374] T 721fcb97fdbc429cbeae5828cad858c0 P c0734e0cf1de44bea1447f3fd21084f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:07:08.848320 22655 raft_consensus.cc:729] T 721fcb97fdbc429cbeae5828cad858c0 P c0734e0cf1de44bea1447f3fd21084f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0734e0cf1de44bea1447f3fd21084f4, State: Initialized, Role: FOLLOWER
I20260808 06:07:08.848763 22655 consensus_queue.cc:260] T 721fcb97fdbc429cbeae5828cad858c0 P c0734e0cf1de44bea1447f3fd21084f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b8cf2d52d5e4ccc86231319427348d5" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 35395 } } peers { permanent_uuid: "c0734e0cf1de44bea1447f3fd21084f4" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35871 } } peers { permanent_uuid: "a9584f1b8a1f4b13abab271fa94b606e" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 45149 } }
I20260808 06:07:08.849418 22612 heartbeater.cc:503] Master 127.1.151.190:44349 was elected leader, sending a full tablet report...
I20260808 06:07:08.849761 22655 ts_tablet_manager.cc:1428] T 721fcb97fdbc429cbeae5828cad858c0 P c0734e0cf1de44bea1447f3fd21084f4: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260808 06:07:08.974395 22364 tablet.cc:2369] T 721fcb97fdbc429cbeae5828cad858c0 P 0b8cf2d52d5e4ccc86231319427348d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 06:07:08.981338 22150 tablet.cc:2369] T 721fcb97fdbc429cbeae5828cad858c0 P a9584f1b8a1f4b13abab271fa94b606e: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 06:07:08 dist-test-slave-zj8q krb5kdc[20271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169228, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for HTTP/127.1.151.148@KRBTEST.COM
W20260808 06:07:09.002403 22615 tablet.cc:2369] T 721fcb97fdbc429cbeae5828cad858c0 P c0734e0cf1de44bea1447f3fd21084f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 06:07:09.031574 22658 raft_consensus.cc:482] T 721fcb97fdbc429cbeae5828cad858c0 P a9584f1b8a1f4b13abab271fa94b606e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:07:09.031839 22658 raft_consensus.cc:504] T 721fcb97fdbc429cbeae5828cad858c0 P a9584f1b8a1f4b13abab271fa94b606e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b8cf2d52d5e4ccc86231319427348d5" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 35395 } } peers { permanent_uuid: "c0734e0cf1de44bea1447f3fd21084f4" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35871 } } peers { permanent_uuid: "a9584f1b8a1f4b13abab271fa94b606e" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 45149 } }
I20260808 06:07:09.033289 22658 leader_election.cc:288] T 721fcb97fdbc429cbeae5828cad858c0 P a9584f1b8a1f4b13abab271fa94b606e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b8cf2d52d5e4ccc86231319427348d5 (127.1.151.130:35395), c0734e0cf1de44bea1447f3fd21084f4 (127.1.151.131:35871)
I20260808 06:07:09.040376 22668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:09.033663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:35395 (local address 127.1.151.129:57075)
0808 06:07:09.034978 (+ 1315us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:09.034992 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:09.035124 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:09.035489 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:09.035492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:09.035509 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:07:09.035659 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:09.035666 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:09.036652 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:09.036655 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:09.038913 (+ 2258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:09.038925 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:09.039872 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:09.039876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:09.039977 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:09.040003 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 06:07:09.040098 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 06:07:09.040183 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1248,"mutex_wait_us":1081,"thread_start_us":122,"threads_started":1}
I20260808 06:07:09.040406 22647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:09.034676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:57075 (local address 127.1.151.130:35395)
0808 06:07:09.034803 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:09.034807 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:09.035075 (+ 268us) server_negotiation.cc:407] Connection header received
0808 06:07:09.035219 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:09.035222 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:09.035276 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:09.035340 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:07:09.035767 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:09.036503 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:09.039030 (+ 2527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:09.039652 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:09.039786 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:09.039830 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:09.040157 (+ 327us) server_negotiation.cc:299] Negotiation successful
0808 06:07:09.040260 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 06:07:09.041535 22667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:09.034074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:35871 (local address 127.1.151.129:60475)
0808 06:07:09.035359 (+ 1285us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:09.035372 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:09.035789 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:09.036144 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:09.036147 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:09.036163 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:07:09.036275 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:09.036280 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:09.037224 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:09.037227 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:09.038459 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:09.038471 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:09.041005 (+ 2534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:09.041009 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:09.041157 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:09.041182 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 06:07:09.041300 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 06:07:09.041373 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1222,"mutex_wait_us":3,"thread_start_us":119,"threads_started":1}
I20260808 06:07:09.041556 22651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:09.034518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:60475 (local address 127.1.151.131:35871)
0808 06:07:09.034821 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:09.034825 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:09.035437 (+ 612us) server_negotiation.cc:407] Connection header received
0808 06:07:09.035907 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:09.035911 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:09.035957 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:09.036004 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:07:09.036383 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:09.037088 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:09.039084 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:09.039653 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:09.039748 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:09.039805 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:09.041353 (+ 1548us) server_negotiation.cc:299] Negotiation successful
0808 06:07:09.041444 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227}
I20260808 06:07:09.041640 22294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "721fcb97fdbc429cbeae5828cad858c0" candidate_uuid: "a9584f1b8a1f4b13abab271fa94b606e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b8cf2d52d5e4ccc86231319427348d5" is_pre_election: true
I20260808 06:07:09.041993 22294 raft_consensus.cc:2454] T 721fcb97fdbc429cbeae5828cad858c0 P 0b8cf2d52d5e4ccc86231319427348d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9584f1b8a1f4b13abab271fa94b606e in term 0.
I20260808 06:07:09.042557 22527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "721fcb97fdbc429cbeae5828cad858c0" candidate_uuid: "a9584f1b8a1f4b13abab271fa94b606e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0734e0cf1de44bea1447f3fd21084f4" is_pre_election: true
I20260808 06:07:09.042618 21981 leader_election.cc:302] T 721fcb97fdbc429cbeae5828cad858c0 P a9584f1b8a1f4b13abab271fa94b606e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b8cf2d52d5e4ccc86231319427348d5, a9584f1b8a1f4b13abab271fa94b606e; no voters:
I20260808 06:07:09.042867 22527 raft_consensus.cc:2454] T 721fcb97fdbc429cbeae5828cad858c0 P c0734e0cf1de44bea1447f3fd21084f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9584f1b8a1f4b13abab271fa94b606e in term 0.
I20260808 06:07:09.043170 22658 raft_consensus.cc:2781] T 721fcb97fdbc429cbeae5828cad858c0 P a9584f1b8a1f4b13abab271fa94b606e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 06:07:09.043272 22658 raft_consensus.cc:482] T 721fcb97fdbc429cbeae5828cad858c0 P a9584f1b8a1f4b13abab271fa94b606e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 06:07:09.043324 22658 raft_consensus.cc:3037] T 721fcb97fdbc429cbeae5828cad858c0 P a9584f1b8a1f4b13abab271fa94b606e [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:07:09.045229 22658 raft_consensus.cc:504] T 721fcb97fdbc429cbeae5828cad858c0 P a9584f1b8a1f4b13abab271fa94b606e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b8cf2d52d5e4ccc86231319427348d5" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 35395 } } peers { permanent_uuid: "c0734e0cf1de44bea1447f3fd21084f4" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35871 } } peers { permanent_uuid: "a9584f1b8a1f4b13abab271fa94b606e" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 45149 } }
I20260808 06:07:09.045634 22658 leader_election.cc:288] T 721fcb97fdbc429cbeae5828cad858c0 P a9584f1b8a1f4b13abab271fa94b606e [CANDIDATE]: Term 1 election: Requested vote from peers 0b8cf2d52d5e4ccc86231319427348d5 (127.1.151.130:35395), c0734e0cf1de44bea1447f3fd21084f4 (127.1.151.131:35871)
I20260808 06:07:09.046132 22527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "721fcb97fdbc429cbeae5828cad858c0" candidate_uuid: "a9584f1b8a1f4b13abab271fa94b606e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0734e0cf1de44bea1447f3fd21084f4"
I20260808 06:07:09.046294 22527 raft_consensus.cc:3037] T 721fcb97fdbc429cbeae5828cad858c0 P c0734e0cf1de44bea1447f3fd21084f4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:07:09.046326 22294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "721fcb97fdbc429cbeae5828cad858c0" candidate_uuid: "a9584f1b8a1f4b13abab271fa94b606e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b8cf2d52d5e4ccc86231319427348d5"
I20260808 06:07:09.046437 22294 raft_consensus.cc:3037] T 721fcb97fdbc429cbeae5828cad858c0 P 0b8cf2d52d5e4ccc86231319427348d5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:07:09.047537 22294 raft_consensus.cc:2454] T 721fcb97fdbc429cbeae5828cad858c0 P 0b8cf2d52d5e4ccc86231319427348d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9584f1b8a1f4b13abab271fa94b606e in term 1.
I20260808 06:07:09.047537 22527 raft_consensus.cc:2454] T 721fcb97fdbc429cbeae5828cad858c0 P c0734e0cf1de44bea1447f3fd21084f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9584f1b8a1f4b13abab271fa94b606e in term 1.
I20260808 06:07:09.047869 21981 leader_election.cc:302] T 721fcb97fdbc429cbeae5828cad858c0 P a9584f1b8a1f4b13abab271fa94b606e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b8cf2d52d5e4ccc86231319427348d5, a9584f1b8a1f4b13abab271fa94b606e; no voters:
I20260808 06:07:09.048218 22658 raft_consensus.cc:2781] T 721fcb97fdbc429cbeae5828cad858c0 P a9584f1b8a1f4b13abab271fa94b606e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:07:09.048420 22658 raft_consensus.cc:686] T 721fcb97fdbc429cbeae5828cad858c0 P a9584f1b8a1f4b13abab271fa94b606e [term 1 LEADER]: Becoming Leader. State: Replica: a9584f1b8a1f4b13abab271fa94b606e, State: Running, Role: LEADER
I20260808 06:07:09.048782 22658 consensus_queue.cc:237] T 721fcb97fdbc429cbeae5828cad858c0 P a9584f1b8a1f4b13abab271fa94b606e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b8cf2d52d5e4ccc86231319427348d5" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 35395 } } peers { permanent_uuid: "c0734e0cf1de44bea1447f3fd21084f4" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35871 } } peers { permanent_uuid: "a9584f1b8a1f4b13abab271fa94b606e" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 45149 } }
I20260808 06:07:09.056387 21837 catalog_manager.cc:5495] T 721fcb97fdbc429cbeae5828cad858c0 P a9584f1b8a1f4b13abab271fa94b606e reported cstate change: term changed from 0 to 1, leader changed from <none> to a9584f1b8a1f4b13abab271fa94b606e (127.1.151.129). New cstate: current_term: 1 leader_uuid: "a9584f1b8a1f4b13abab271fa94b606e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b8cf2d52d5e4ccc86231319427348d5" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 35395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0734e0cf1de44bea1447f3fd21084f4" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9584f1b8a1f4b13abab271fa94b606e" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 45149 } health_report { overall_health: HEALTHY } } }
I20260808 06:07:09.116329 22650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:09.110963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56504 (local address 127.1.151.129:45149)
0808 06:07:09.111178 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:09.111183 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:09.111200 (+ 17us) server_negotiation.cc:407] Connection header received
0808 06:07:09.111303 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:09.111307 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:09.111390 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:09.111509 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:07:09.111945 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:09.112769 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:09.113979 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:09.114324 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:09.114431 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:09.114896 (+ 465us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:07:09.115486 (+ 590us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:07:09.115629 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:09.116030 (+ 401us) server_negotiation.cc:299] Negotiation successful
0808 06:07:09.116094 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 06:07:09.134550 22527 raft_consensus.cc:1261] T 721fcb97fdbc429cbeae5828cad858c0 P c0734e0cf1de44bea1447f3fd21084f4 [term 1 FOLLOWER]: Refusing update from remote peer a9584f1b8a1f4b13abab271fa94b606e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:07:09.134773 22294 raft_consensus.cc:1261] T 721fcb97fdbc429cbeae5828cad858c0 P 0b8cf2d52d5e4ccc86231319427348d5 [term 1 FOLLOWER]: Refusing update from remote peer a9584f1b8a1f4b13abab271fa94b606e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:07:09.135537 22671 consensus_queue.cc:1035] T 721fcb97fdbc429cbeae5828cad858c0 P a9584f1b8a1f4b13abab271fa94b606e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b8cf2d52d5e4ccc86231319427348d5" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 35395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:07:09.135993 22671 consensus_queue.cc:1035] T 721fcb97fdbc429cbeae5828cad858c0 P a9584f1b8a1f4b13abab271fa94b606e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0734e0cf1de44bea1447f3fd21084f4" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:07:09.149284 22675 mvcc.cc:205] Tried to move back new op lower bound from 7316149162526195712 to 7316149162187378688. Current Snapshot: MvccSnapshot[applied={T|T < 7316149162526195712}]
I20260808 06:07:09.163148 21839 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60322:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:07:09.163370 21839 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60322:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:07:09.166502 21839 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3eb419af59bc4da6a8074f1dc3169721: Sending DeleteTablet for 3 replicas of tablet 721fcb97fdbc429cbeae5828cad858c0
I20260808 06:07:09.167433 22055 tablet_service.cc:1514] Processing DeleteTablet for tablet 721fcb97fdbc429cbeae5828cad858c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:07:09 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:56496
I20260808 06:07:09.168169 22489 tablet_service.cc:1514] Processing DeleteTablet for tablet 721fcb97fdbc429cbeae5828cad858c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:07:09 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:59550
I20260808 06:07:09.168615 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 21908
I20260808 06:07:09.168660 22264 tablet_service.cc:1514] Processing DeleteTablet for tablet 721fcb97fdbc429cbeae5828cad858c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:07:09 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:60116
I20260808 06:07:09.179507 22689 tablet_replica.cc:330] T 721fcb97fdbc429cbeae5828cad858c0 P c0734e0cf1de44bea1447f3fd21084f4: stopping tablet replica
I20260808 06:07:09.180008 22689 raft_consensus.cc:2229] T 721fcb97fdbc429cbeae5828cad858c0 P c0734e0cf1de44bea1447f3fd21084f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:07:09.180475 22689 raft_consensus.cc:2258] T 721fcb97fdbc429cbeae5828cad858c0 P c0734e0cf1de44bea1447f3fd21084f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:07:09.181087 22678 mvcc.cc:205] Tried to move back new op lower bound from 7316149162526195712 to 7316149162187378688. Current Snapshot: MvccSnapshot[applied={T|T < 7316149162526195712 or (T in {7316149162526195712})}]
I20260808 06:07:09.183765 22690 tablet_replica.cc:330] T 721fcb97fdbc429cbeae5828cad858c0 P 0b8cf2d52d5e4ccc86231319427348d5: stopping tablet replica
I20260808 06:07:09.184742 22690 raft_consensus.cc:2229] T 721fcb97fdbc429cbeae5828cad858c0 P 0b8cf2d52d5e4ccc86231319427348d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:07:09.185108 22690 raft_consensus.cc:2258] T 721fcb97fdbc429cbeae5828cad858c0 P 0b8cf2d52d5e4ccc86231319427348d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:07:09.188446 22689 ts_tablet_manager.cc:1905] T 721fcb97fdbc429cbeae5828cad858c0 P c0734e0cf1de44bea1447f3fd21084f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:07:09.190815 22690 ts_tablet_manager.cc:1905] T 721fcb97fdbc429cbeae5828cad858c0 P 0b8cf2d52d5e4ccc86231319427348d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:07:09.191411 22689 ts_tablet_manager.cc:1918] T 721fcb97fdbc429cbeae5828cad858c0 P c0734e0cf1de44bea1447f3fd21084f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:07:09.191628 22689 log.cc:1196] T 721fcb97fdbc429cbeae5828cad858c0 P c0734e0cf1de44bea1447f3fd21084f4: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-2/wal/wals/721fcb97fdbc429cbeae5828cad858c0
I20260808 06:07:09.192281 22689 ts_tablet_manager.cc:1939] T 721fcb97fdbc429cbeae5828cad858c0 P c0734e0cf1de44bea1447f3fd21084f4: Deleting consensus metadata
I20260808 06:07:09.193567 22690 ts_tablet_manager.cc:1918] T 721fcb97fdbc429cbeae5828cad858c0 P 0b8cf2d52d5e4ccc86231319427348d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:07:09.193681 21812 catalog_manager.cc:4841] TS c0734e0cf1de44bea1447f3fd21084f4 (127.1.151.131:35871): tablet 721fcb97fdbc429cbeae5828cad858c0 (table test-table [id=76d5e7b4fb0b4f6295f8af2e4ca59c2b]) successfully deleted
I20260808 06:07:09.197590 22690 log.cc:1196] T 721fcb97fdbc429cbeae5828cad858c0 P 0b8cf2d52d5e4ccc86231319427348d5: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786169070948234-1630-0/minicluster-data/ts-1/wal/wals/721fcb97fdbc429cbeae5828cad858c0
I20260808 06:07:09.197937 22690 ts_tablet_manager.cc:1939] T 721fcb97fdbc429cbeae5828cad858c0 P 0b8cf2d52d5e4ccc86231319427348d5: Deleting consensus metadata
W20260808 06:07:09.199437 21815 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.151.129:45149 (error 108)
I20260808 06:07:09.199448 21812 catalog_manager.cc:4841] TS 0b8cf2d52d5e4ccc86231319427348d5 (127.1.151.130:35395): tablet 721fcb97fdbc429cbeae5828cad858c0 (table test-table [id=76d5e7b4fb0b4f6295f8af2e4ca59c2b]) successfully deleted
I20260808 06:07:09.200309 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 22160
I20260808 06:07:09.234877 22648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:09.234479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:45149 (local address 127.0.0.1:56516)
0808 06:07:09.234695 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:09.234782 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.129:45149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
W20260808 06:07:09.235236 21815 catalog_manager.cc:4584] TS a9584f1b8a1f4b13abab271fa94b606e (127.1.151.129:45149): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 721fcb97fdbc429cbeae5828cad858c0: Network error: Client connection negotiation failed: client connection to 127.1.151.129:45149: connect: Connection refused (error 111)
I20260808 06:07:09.263815 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 22369
I20260808 06:07:09.278697 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 21784
I20260808 06:07:09.290913 1630 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 06:07:09.635066 1630 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 06:07:09.635177 1630 mini_ranger.cc:66] Stopping Ranger...
I20260808 06:07:09.979283 1630 mini_ranger.cc:68] Stopped Ranger
2026-08-08 06:07:09.979 UTC [20315] LOG: received smart shutdown request
2026-08-08 06:07:09.983 UTC [20315] LOG: background worker "logical replication launcher" (PID 20331) exited with exit code 1
2026-08-08 06:07:09.983 UTC [20322] LOG: shutting down
2026-08-08 06:07:10.126 UTC [20315] LOG: database system is shut down
2026-08-08T06:07:10Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (47485 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:07:10 dist-test-slave-zj8q krb5kdc[22707](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:07:10 dist-test-slave-zj8q krb5kdc[22707](info): set up 2 sockets
Aug 08 06:07:10 dist-test-slave-zj8q krb5kdc[22707](info): commencing operation
krb5kdc: starting...
W20260808 06:07:11.365015 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:07:11 dist-test-slave-zj8q krb5kdc[22707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:07:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:07:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.151.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
I20260808 06:07:11.481894 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:39967
--webserver_interface=127.1.151.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:46237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:39967,127.1.151.189:46301,127.1.151.188:46179
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:07:11.589777 22724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:07:11.590097 22724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:07:11.590197 22724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:07:11.594146 22724 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:07:11.594223 22724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:07:11.594249 22724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:07:11.594270 22724 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:07:11.594301 22724 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:07:11.599489 22724 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:46237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:39967,127.1.151.189:46301,127.1.151.188:46179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:39967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:07:11.600492 22724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:07:11.601478 22724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:07:11.609591 22733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:11.609637 22731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:11.609792 22724 server_base.cc:1034] running on GCE node
W20260808 06:07:11.609648 22730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:11.610473 22724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:07:11.611485 22724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:07:11.612640 22724 hybrid_clock.cc:648] HybridClock initialized: now 1786169231612622 us; error 30 us; skew 500 ppm
Aug 08 06:07:11 dist-test-slave-zj8q krb5kdc[22707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169231, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:07:11.615283 22724 init.cc:377] Logged in from keytab as kudu/127.1.151.190@KRBTEST.COM (short username kudu)
I20260808 06:07:11.616073 22724 webserver.cc:460] Webserver started at http://127.1.151.190:34665/ using document root <none> and password file <none>
I20260808 06:07:11.616555 22724 fs_manager.cc:359] Metadata directory not provided
I20260808 06:07:11.616609 22724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:07:11.616780 22724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:07:11.618726 22724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "7e3e6a8eaa824d29a60a698db52f8c05"
format_stamp: "Formatted at 2026-08-08 06:07:11 on dist-test-slave-zj8q"
server_key: "418d2e506d1c6e87329295ccfc7f66d8"
server_key_iv: "1e44f8b90765cc472efd6388e89db797"
server_key_version: "encryptionkey@0"
I20260808 06:07:11.619230 22724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "7e3e6a8eaa824d29a60a698db52f8c05"
format_stamp: "Formatted at 2026-08-08 06:07:11 on dist-test-slave-zj8q"
server_key: "418d2e506d1c6e87329295ccfc7f66d8"
server_key_iv: "1e44f8b90765cc472efd6388e89db797"
server_key_version: "encryptionkey@0"
I20260808 06:07:11.622589 22724 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 06:07:11.624774 22748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:11.625885 22724 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:07:11.625982 22724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "7e3e6a8eaa824d29a60a698db52f8c05"
format_stamp: "Formatted at 2026-08-08 06:07:11 on dist-test-slave-zj8q"
server_key: "418d2e506d1c6e87329295ccfc7f66d8"
server_key_iv: "1e44f8b90765cc472efd6388e89db797"
server_key_version: "encryptionkey@0"
I20260808 06:07:11.626061 22724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:07:11.660073 22724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:07:11.663090 22724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:07:11.663239 22724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:07:11.670050 22724 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:39967
I20260808 06:07:11.670064 22823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:39967 every 8 connection(s)
I20260808 06:07:11.670971 22724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:07:11.673610 22824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:07:11.677299 22824 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39967 } has no permanent_uuid. Determining permanent_uuid...
I20260808 06:07:11.677611 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 22724
I20260808 06:07:11.677745 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:07:11.678042 1630 external_mini_cluster.cc:1428] Setting key 6ba7047a473644ad18b8bfe6d6554cf2
WARNING: no policy specified for kudu/127.1.151.189@KRBTEST.COM; defaulting to no policy
Aug 08 06:07:11 dist-test-slave-zj8q krb5kdc[22707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169231, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.189@KRBTEST.COM" created.
I20260808 06:07:11.692188 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:11.679751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39967 (local address 127.0.0.1:52436)
0808 06:07:11.680417 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:11.680451 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:11.681219 (+ 768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:11.681859 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:11.681865 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:11.682319 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:07:11.682651 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:11.682668 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:11.683795 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:11.683799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:11.684959 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:11.684971 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:11.685577 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:11.685582 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:11.685716 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:11.686269 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:07:11.686291 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:07:11.687814 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:07:11.689492 (+ 1678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:11.689500 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:11.689507 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:11.689727 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:11.690055 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:11.690058 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:11.690059 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:11.690208 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:11.690763 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:07:11.690770 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:07:11.690925 (+ 155us) client_negotiation.cc:769] Sending connection context
0808 06:07:11.691091 (+ 166us) client_negotiation.cc:240] Negotiation successful
0808 06:07:11.691456 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"mutex_wait_us":80,"thread_start_us":147,"threads_started":1}
I20260808 06:07:11.692184 22828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:11.679751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52436 (local address 127.1.151.190:39967)
0808 06:07:11.680317 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:11.680335 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:11.680544 (+ 209us) server_negotiation.cc:407] Connection header received
0808 06:07:11.681491 (+ 947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:11.681495 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:11.681662 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:11.681805 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:11.682791 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:11.683665 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:11.685064 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:11.685415 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:11.685536 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:11.687959 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:11.687965 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:11.687974 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:11.687996 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:11.689371 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:11.689829 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:11.689835 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:11.689836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:11.689971 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:11.690296 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:11.690298 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:11.690299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:11.690646 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:11.690748 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:11.691163 (+ 415us) server_negotiation.cc:299] Negotiation successful
0808 06:07:11.691365 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":433,"spinlock_wait_cycles":7424,"thread_start_us":147,"threads_started":1}
I20260808 06:07:11.694502 22824 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.151.189" port: 46301 } has no permanent_uuid. Determining permanent_uuid...
I20260808 06:07:11.695200 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:11.694956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.189:46301 (local address 127.0.0.1:58708)
0808 06:07:11.695065 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:11.695132 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.189:46301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 06:07:11.695423 22752 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.151.189:46301: connect: Connection refused (error 111)
I20260808 06:07:11.696300 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:11.696035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.189:46301 (local address 127.0.0.1:58712)
0808 06:07:11.696197 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:11.696245 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.189:46301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
W20260808 06:07:11.696465 22824 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.151.189:46301: Network error: Client connection negotiation failed: client connection to 127.1.151.189:46301: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.189.keytab.
Entry for principal kudu/127.1.151.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.189.keytab.
I20260808 06:07:11.705340 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.189:46301
--webserver_interface=127.1.151.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.189.keytab
--principal=kudu/127.1.151.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:46237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:39967,127.1.151.189:46301,127.1.151.188:46179
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 06:07:11.745715 22824 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.151.189" port: 46301 } attempt: 1
I20260808 06:07:11.746660 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:11.746343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.189:46301 (local address 127.0.0.1:58720)
0808 06:07:11.746501 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:11.746575 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.189:46301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 06:07:11.747373 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:11.747152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.189:46301 (local address 127.0.0.1:58726)
0808 06:07:11.747258 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:11.747306 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.189:46301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 06:07:11.747496 22824 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.151.189:46301: Network error: Client connection negotiation failed: client connection to 127.1.151.189:46301: connect: Connection refused (error 111)
I20260808 06:07:11.815707 22824 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.151.189" port: 46301 } attempt: 2
I20260808 06:07:11.816547 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:11.816245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.189:46301 (local address 127.0.0.1:58742)
0808 06:07:11.816407 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:11.816469 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.189:46301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 06:07:11.817231 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:11.817019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.189:46301 (local address 127.0.0.1:58752)
0808 06:07:11.817126 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:11.817171 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.189:46301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 06:07:11.817344 22824 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.151.189:46301: Network error: Client connection negotiation failed: client connection to 127.1.151.189:46301: connect: Connection refused (error 111)
W20260808 06:07:11.818432 22833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:07:11.818668 22833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:07:11.818737 22833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:07:11.822662 22833 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:07:11.822710 22833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:07:11.822741 22833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:07:11.822772 22833 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:07:11.822800 22833 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:07:11.827972 22833 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:46237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:39967,127.1.151.189:46301,127.1.151.188:46179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.189.keytab
--principal=kudu/127.1.151.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.189:46301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.151.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:07:11.829001 22833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:07:11.830045 22833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:07:11.838328 22844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:11.838382 22833 server_base.cc:1034] running on GCE node
W20260808 06:07:11.838383 22842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:11.838333 22841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:11.839160 22833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:07:11.840181 22833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:07:11.841352 22833 hybrid_clock.cc:648] HybridClock initialized: now 1786169231841333 us; error 29 us; skew 500 ppm
Aug 08 06:07:11 dist-test-slave-zj8q krb5kdc[22707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169231, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:07:11.844084 22833 init.cc:377] Logged in from keytab as kudu/127.1.151.189@KRBTEST.COM (short username kudu)
I20260808 06:07:11.844899 22833 webserver.cc:460] Webserver started at http://127.1.151.189:37371/ using document root <none> and password file <none>
I20260808 06:07:11.845431 22833 fs_manager.cc:359] Metadata directory not provided
I20260808 06:07:11.845498 22833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:07:11.845702 22833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:07:11.847754 22833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-1/data/instance:
uuid: "1c1f3d7ee0fd41329b59a8d18cc32452"
format_stamp: "Formatted at 2026-08-08 06:07:11 on dist-test-slave-zj8q"
server_key: "94289a5b86610994df5b4235aac679bf"
server_key_iv: "afee3280b5fdbfd0650a3a16a7decf76"
server_key_version: "encryptionkey@0"
I20260808 06:07:11.848301 22833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-1/wal/instance:
uuid: "1c1f3d7ee0fd41329b59a8d18cc32452"
format_stamp: "Formatted at 2026-08-08 06:07:11 on dist-test-slave-zj8q"
server_key: "94289a5b86610994df5b4235aac679bf"
server_key_iv: "afee3280b5fdbfd0650a3a16a7decf76"
server_key_version: "encryptionkey@0"
I20260808 06:07:11.851876 22833 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 06:07:11.854208 22854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:11.855448 22833 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:07:11.855571 22833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-1/wal
uuid: "1c1f3d7ee0fd41329b59a8d18cc32452"
format_stamp: "Formatted at 2026-08-08 06:07:11 on dist-test-slave-zj8q"
server_key: "94289a5b86610994df5b4235aac679bf"
server_key_iv: "afee3280b5fdbfd0650a3a16a7decf76"
server_key_version: "encryptionkey@0"
I20260808 06:07:11.855687 22833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:07:11.871487 22833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:07:11.874632 22833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:07:11.874812 22833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:07:11.881675 22833 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.189:46301
I20260808 06:07:11.881718 22954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.189:46301 every 8 connection(s)
I20260808 06:07:11.882609 22833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-1/data/info.pb
I20260808 06:07:11.885264 22955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:07:11.889124 22955 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39967 } has no permanent_uuid. Determining permanent_uuid...
I20260808 06:07:11.891786 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 22833
I20260808 06:07:11.891917 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-1/wal/instance
I20260808 06:07:11.892230 1630 external_mini_cluster.cc:1428] Setting key be02b071ac4b23bef571681f80ec5395
Aug 08 06:07:11 dist-test-slave-zj8q krb5kdc[22707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169231, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.189@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.151.188@KRBTEST.COM; defaulting to no policy
I20260808 06:07:11.902421 22828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:11.890959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52438 (local address 127.1.151.190:39967)
0808 06:07:11.891086 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:11.891089 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:11.891932 (+ 843us) server_negotiation.cc:407] Connection header received
0808 06:07:11.893080 (+ 1148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:11.893083 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:11.893134 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:11.893208 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:11.894629 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:11.895184 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:11.896269 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:11.896505 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:11.896596 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:11.898976 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:11.898979 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:11.898981 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:11.899012 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:11.900399 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:11.900913 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:11.900916 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:11.900917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:11.901002 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:11.901319 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:11.901322 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:11.901323 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:11.901498 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:11.901592 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:11.902082 (+ 490us) server_negotiation.cc:299] Negotiation successful
0808 06:07:11.902234 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.188@KRBTEST.COM" created.
I20260808 06:07:11.903229 22958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:11.891295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39967 (local address 127.0.0.1:52438)
0808 06:07:11.891773 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:11.891831 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:11.892851 (+ 1020us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:11.893391 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:11.893399 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:11.893822 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:07:11.894479 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:11.894494 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:11.895310 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:11.895313 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:11.896153 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:11.896163 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:11.896645 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:11.896649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:11.896809 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:11.897417 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:07:11.897433 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:07:11.898821 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:07:11.900507 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:11.900513 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:11.900526 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:11.900821 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:11.901087 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:11.901090 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:11.901093 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:11.901233 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:11.901601 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:07:11.901607 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:07:11.901848 (+ 241us) client_negotiation.cc:769] Sending connection context
0808 06:07:11.902005 (+ 157us) client_negotiation.cc:240] Negotiation successful
0808 06:07:11.902326 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":120,"threads_started":1}
I20260808 06:07:11.904627 22955 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.151.189" port: 46301 } has no permanent_uuid. Determining permanent_uuid...
I20260808 06:07:11.908488 22824 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.151.189" port: 46301 } attempt: 3
Aug 08 06:07:11 dist-test-slave-zj8q krb5kdc[22707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169231, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.189@KRBTEST.COM for kudu/127.1.151.189@KRBTEST.COM
Aug 08 06:07:11 dist-test-slave-zj8q krb5kdc[22707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169231, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for kudu/127.1.151.189@KRBTEST.COM
I20260808 06:07:11.914693 22958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:11.905085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.189:46301 (local address 127.0.0.1:58766)
0808 06:07:11.905190 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:11.905203 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:11.905267 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:11.905855 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:11.905858 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:11.906012 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:07:11.906094 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:11.906113 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:11.907013 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:11.907016 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:11.907537 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:11.907545 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:11.908063 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:11.908067 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:11.908204 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:11.908674 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:07:11.908688 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:07:11.910010 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:07:11.912549 (+ 2539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:11.912553 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:11.912554 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:11.913310 (+ 756us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:11.913587 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:11.913590 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:11.913592 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:11.913682 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:11.914216 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:07:11.914219 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:07:11.914300 (+ 81us) client_negotiation.cc:769] Sending connection context
0808 06:07:11.914389 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 06:07:11.914510 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":36}
I20260808 06:07:11.914848 22964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:11.905251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58766 (local address 127.1.151.189:46301)
0808 06:07:11.905548 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:11.905556 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:11.905573 (+ 17us) server_negotiation.cc:407] Connection header received
0808 06:07:11.905610 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:11.905614 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:11.905746 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:11.905854 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:11.906219 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:11.906911 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:11.907641 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:11.907967 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:11.908055 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:11.910197 (+ 2142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:11.910209 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:11.910222 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:11.910249 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:11.912452 (+ 2203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:11.913416 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:11.913421 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:11.913422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:11.913500 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:11.913841 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:11.913844 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:11.913846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:11.914072 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:11.914340 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:11.914477 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 06:07:11.914620 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":206,"thread_start_us":124,"threads_started":1}
I20260808 06:07:11.916559 22955 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.151.188" port: 46179 } has no permanent_uuid. Determining permanent_uuid...
I20260808 06:07:11.917225 22958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:11.917007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:46179 (local address 127.0.0.1:45206)
0808 06:07:11.917083 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:11.917163 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 06:07:11.917413 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:11.908840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.189:46301 (local address 127.0.0.1:58782)
0808 06:07:11.908998 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:11.909011 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:11.909083 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:11.909447 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:11.909450 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:11.909611 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:07:11.910172 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:11.910184 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:11.911005 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:11.911008 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:11.911628 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:11.911640 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:11.912212 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:11.912215 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:11.912297 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:11.912731 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:07:11.912742 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:07:11.914035 (+ 1293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:07:11.915857 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:11.915860 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:11.915861 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:11.916077 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:11.916350 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:11.916354 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:11.916356 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:11.916457 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:11.916911 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:07:11.916914 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:07:11.917016 (+ 102us) client_negotiation.cc:769] Sending connection context
0808 06:07:11.917126 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 06:07:11.917256 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260808 06:07:11.917430 22966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:11.908911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58782 (local address 127.1.151.189:46301)
0808 06:07:11.909219 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:11.909228 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:11.909242 (+ 14us) server_negotiation.cc:407] Connection header received
0808 06:07:11.909277 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:11.909280 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:11.909320 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:11.909415 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:11.910277 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:11.910905 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:11.911762 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:11.912106 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:11.912218 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:11.914198 (+ 1980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:11.914201 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:11.914204 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:11.914227 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:11.915740 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:11.916163 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:11.916166 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:11.916168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:11.916254 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:11.916535 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:11.916538 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:11.916539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:11.916804 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:11.916909 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:11.917190 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 06:07:11.917295 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":98,"threads_started":1}
W20260808 06:07:11.917409 22860 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.151.188:46179: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.188.keytab.
Entry for principal kudu/127.1.151.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.188.keytab.
I20260808 06:07:11.918015 22824 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.151.188" port: 46179 } has no permanent_uuid. Determining permanent_uuid...
I20260808 06:07:11.918421 22958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:11.918196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:46179 (local address 127.0.0.1:45208)
0808 06:07:11.918310 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:11.918361 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 06:07:11.918720 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:11.918415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:46179 (local address 127.0.0.1:45210)
0808 06:07:11.918565 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:11.918613 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260808 06:07:11.918998 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.188:46179
--webserver_interface=127.1.151.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.188.keytab
--principal=kudu/127.1.151.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:46237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:39967,127.1.151.189:46301,127.1.151.188:46179
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 06:07:11.919425 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:11.919166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:46179 (local address 127.0.0.1:45226)
0808 06:07:11.919312 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:11.919364 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260808 06:07:11.919548 22824 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.151.188:46179: Network error: Client connection negotiation failed: client connection to 127.1.151.188:46179: connect: Connection refused (error 111)
W20260808 06:07:11.920387 22955 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.151.188:46179: Network error: Client connection negotiation failed: client connection to 127.1.151.188:46179: connect: Connection refused (error 111)
I20260808 06:07:11.950699 22824 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.151.188" port: 46179 } attempt: 1
I20260808 06:07:11.951545 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:11.951252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:46179 (local address 127.0.0.1:45230)
0808 06:07:11.951398 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:11.951460 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 06:07:11.952235 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:11.952031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:46179 (local address 127.0.0.1:45240)
0808 06:07:11.952129 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:11.952175 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 06:07:11.952373 22824 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.151.188:46179: Network error: Client connection negotiation failed: client connection to 127.1.151.188:46179: connect: Connection refused (error 111)
I20260808 06:07:11.969664 22955 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.151.188" port: 46179 } attempt: 1
I20260808 06:07:11.970561 22958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:11.970239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:46179 (local address 127.0.0.1:45252)
0808 06:07:11.970400 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:11.970472 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 06:07:11.971292 22958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:11.971075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:46179 (local address 127.0.0.1:45262)
0808 06:07:11.971171 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:11.971225 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 06:07:11.971414 22955 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.151.188:46179: Network error: Client connection negotiation failed: client connection to 127.1.151.188:46179: connect: Connection refused (error 111)
I20260808 06:07:12.027510 22824 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.151.188" port: 46179 } attempt: 2
I20260808 06:07:12.028342 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:12.028039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:46179 (local address 127.0.0.1:45270)
0808 06:07:12.028195 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:12.028258 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 06:07:12.029054 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:12.028847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:46179 (local address 127.0.0.1:45282)
0808 06:07:12.028948 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:12.028993 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 06:07:12.029191 22824 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.151.188:46179: Network error: Client connection negotiation failed: client connection to 127.1.151.188:46179: connect: Connection refused (error 111)
W20260808 06:07:12.030534 22969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:07:12.030776 22969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:07:12.030814 22969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:07:12.034789 22969 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:07:12.034839 22969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:07:12.034857 22969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:07:12.034873 22969 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:07:12.034891 22969 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:07:12.039559 22955 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.151.188" port: 46179 } attempt: 2
I20260808 06:07:12.040069 22969 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:46237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:39967,127.1.151.189:46301,127.1.151.188:46179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.188.keytab
--principal=kudu/127.1.151.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.188:46179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.151.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:07:12.040264 22958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:12.039983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:46179 (local address 127.0.0.1:45298)
0808 06:07:12.040124 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:12.040187 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 06:07:12.040891 22958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:12.040683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:46179 (local address 127.0.0.1:45314)
0808 06:07:12.040777 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:12.040828 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 06:07:12.041028 22955 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.151.188:46179: Network error: Client connection negotiation failed: client connection to 127.1.151.188:46179: connect: Connection refused (error 111)
I20260808 06:07:12.041246 22969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:07:12.042191 22969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:07:12.050117 22975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:12.050100 22979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:12.050102 22976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:12.050446 22969 server_base.cc:1034] running on GCE node
I20260808 06:07:12.051025 22969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:07:12.052021 22969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:07:12.053218 22969 hybrid_clock.cc:648] HybridClock initialized: now 1786169232053202 us; error 23 us; skew 500 ppm
Aug 08 06:07:12 dist-test-slave-zj8q krb5kdc[22707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169232, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:07:12.055948 22969 init.cc:377] Logged in from keytab as kudu/127.1.151.188@KRBTEST.COM (short username kudu)
I20260808 06:07:12.056757 22969 webserver.cc:460] Webserver started at http://127.1.151.188:43931/ using document root <none> and password file <none>
I20260808 06:07:12.057278 22969 fs_manager.cc:359] Metadata directory not provided
I20260808 06:07:12.057344 22969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:07:12.057546 22969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:07:12.059677 22969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-2/data/instance:
uuid: "f6052fe3c96345cda502a5f6f74911a6"
format_stamp: "Formatted at 2026-08-08 06:07:12 on dist-test-slave-zj8q"
server_key: "dfeda5c4daccb748b0c1a2205844efb1"
server_key_iv: "80cfd49c797acab6f4b5e8e4aaf6cd74"
server_key_version: "encryptionkey@0"
I20260808 06:07:12.060227 22969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-2/wal/instance:
uuid: "f6052fe3c96345cda502a5f6f74911a6"
format_stamp: "Formatted at 2026-08-08 06:07:12 on dist-test-slave-zj8q"
server_key: "dfeda5c4daccb748b0c1a2205844efb1"
server_key_iv: "80cfd49c797acab6f4b5e8e4aaf6cd74"
server_key_version: "encryptionkey@0"
I20260808 06:07:12.064030 22969 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260808 06:07:12.066318 22988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:12.067438 22969 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 06:07:12.067548 22969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-2/wal
uuid: "f6052fe3c96345cda502a5f6f74911a6"
format_stamp: "Formatted at 2026-08-08 06:07:12 on dist-test-slave-zj8q"
server_key: "dfeda5c4daccb748b0c1a2205844efb1"
server_key_iv: "80cfd49c797acab6f4b5e8e4aaf6cd74"
server_key_version: "encryptionkey@0"
I20260808 06:07:12.067628 22969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:07:12.086799 22969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:07:12.089673 22969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:07:12.089831 22969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:07:12.096728 22969 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.188:46179
I20260808 06:07:12.096748 23067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.188:46179 every 8 connection(s)
I20260808 06:07:12.097685 22969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-2/data/info.pb
I20260808 06:07:12.100375 23068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:07:12.104027 23068 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39967 } has no permanent_uuid. Determining permanent_uuid...
I20260808 06:07:12.104919 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 22969
I20260808 06:07:12.105024 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786169070948234-1630-0/minicluster-data/master-2/wal/instance
I20260808 06:07:12.105293 1630 external_mini_cluster.cc:1428] Setting key f5c78feef0e69d629aeb880a726ec59b
Aug 08 06:07:12 dist-test-slave-zj8q krb5kdc[22707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169232, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.188@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
Aug 08 06:07:12 dist-test-slave-zj8q krb5kdc[22707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.188@KRBTEST.COM
I20260808 06:07:12.116943 22828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:12.105920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52448 (local address 127.1.151.190:39967)
0808 06:07:12.106048 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:12.106053 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:12.107051 (+ 998us) server_negotiation.cc:407] Connection header received
0808 06:07:12.107838 (+ 787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:12.107842 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:12.107891 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:12.107973 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:12.108894 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:12.109408 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:12.110500 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:12.110749 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:12.110828 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:12.113237 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:12.113241 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:12.113244 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:12.113273 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:12.114830 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:12.115408 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:12.115411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:12.115412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:12.115479 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:12.115793 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:12.115797 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:12.115799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:12.116017 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:12.116123 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:12.116654 (+ 531us) server_negotiation.cc:299] Negotiation successful
0808 06:07:12.116777 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 06:07:12.117863 23074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:12.106405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39967 (local address 127.0.0.1:52448)
0808 06:07:12.106898 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:12.106947 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:12.107629 (+ 682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:12.108031 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:12.108036 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:12.108462 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:07:12.108749 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:12.108762 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:12.109509 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:12.109512 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:12.110391 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:12.110401 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:12.110883 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:12.110887 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:12.111069 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:12.111631 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:07:12.111648 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:07:12.113089 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:07:12.114938 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:12.114946 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:12.114956 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:12.115324 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:12.115544 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:12.115546 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:12.115547 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:12.115691 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:12.116115 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:07:12.116120 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:07:12.116414 (+ 294us) client_negotiation.cc:769] Sending connection context
0808 06:07:12.116598 (+ 184us) client_negotiation.cc:240] Negotiation successful
0808 06:07:12.116896 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"spinlock_wait_cycles":26240,"thread_start_us":172,"threads_started":1}
I20260808 06:07:12.117882 23073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:12.106405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45326 (local address 127.1.151.188:46179)
0808 06:07:12.106845 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:12.106862 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:12.106902 (+ 40us) server_negotiation.cc:407] Connection header received
0808 06:07:12.107618 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:12.107642 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:12.107873 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:12.108027 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:12.108749 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:12.109714 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:12.110610 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:12.111022 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:12.111127 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:12.113708 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:12.113713 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:12.113722 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:12.113742 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:12.115332 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:12.115891 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:12.115896 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:12.115897 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:12.115966 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:12.116220 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:12.116223 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:12.116224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:12.116495 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:12.116692 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:12.116901 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 06:07:12.117016 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"spinlock_wait_cycles":3328,"thread_start_us":132,"threads_started":1}
I20260808 06:07:12.119125 23068 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.151.189" port: 46301 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 06:07:12 dist-test-slave-zj8q krb5kdc[22707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169232, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.188@KRBTEST.COM for kudu/127.1.151.189@KRBTEST.COM
Aug 08 06:07:12 dist-test-slave-zj8q krb5kdc[22707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:07:12.127300 23074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:12.119558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.189:46301 (local address 127.0.0.1:58788)
0808 06:07:12.119661 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:12.119673 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:12.119740 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:12.120157 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:12.120159 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:12.120321 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:07:12.120430 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:12.120436 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:12.121332 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:12.121335 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:12.121946 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:12.121956 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:12.122490 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:12.122494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:12.122586 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:12.122966 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:07:12.122976 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:07:12.124107 (+ 1131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:07:12.125952 (+ 1845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:12.125955 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:12.125957 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:12.126136 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:12.126413 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:12.126416 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:12.126418 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:12.126510 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:12.126877 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:07:12.126880 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:07:12.126980 (+ 100us) client_negotiation.cc:769] Sending connection context
0808 06:07:12.127084 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 06:07:12.127191 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 06:07:12.127429 22966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:12.119826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58788 (local address 127.1.151.189:46301)
0808 06:07:12.119938 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:12.119943 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:12.119957 (+ 14us) server_negotiation.cc:407] Connection header received
0808 06:07:12.120000 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:12.120003 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:12.120049 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:12.120126 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:12.120552 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:12.121233 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:12.122061 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:12.122394 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:12.122471 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:12.124234 (+ 1763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:12.124237 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:12.124239 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:12.124262 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:12.125851 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:12.126238 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:12.126241 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:12.126243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:12.126328 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:12.126593 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:12.126596 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:12.126597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:12.126783 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:12.126880 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:12.127153 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 06:07:12.127281 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 06:07:12.128039 23068 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.151.188" port: 46179 } has no permanent_uuid. Determining permanent_uuid...
I20260808 06:07:12.128273 22828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:12.119940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52454 (local address 127.1.151.190:39967)
0808 06:07:12.120042 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:12.120046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:12.120089 (+ 43us) server_negotiation.cc:407] Connection header received
0808 06:07:12.120220 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:12.120223 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:12.120264 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:12.120338 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:12.120959 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:12.121482 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:12.122254 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:12.122467 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:12.122565 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:12.124826 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:12.124830 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:12.124832 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:12.124857 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:12.126351 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:12.126758 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:12.126761 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:12.126763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:12.126842 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:12.127096 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:12.127099 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:12.127101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:12.127305 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:12.127655 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:12.127983 (+ 328us) server_negotiation.cc:299] Negotiation successful
0808 06:07:12.128088 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 06:07:12.128353 22824 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.151.188" port: 46179 } attempt: 3
I20260808 06:07:12.132167 22955 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.151.188" port: 46179 } attempt: 3
Aug 08 06:07:12 dist-test-slave-zj8q krb5kdc[22707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169231, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for kudu/127.1.151.188@KRBTEST.COM
Aug 08 06:07:12 dist-test-slave-zj8q krb5kdc[22707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169232, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.188@KRBTEST.COM for kudu/127.1.151.188@KRBTEST.COM
Aug 08 06:07:12 dist-test-slave-zj8q krb5kdc[22707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.189@KRBTEST.COM
I20260808 06:07:12.137923 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:12.128687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:46179 (local address 127.0.0.1:45340)
0808 06:07:12.128843 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:12.128855 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:12.128914 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:12.129749 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:12.129753 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:12.129922 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:07:12.130019 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:12.130026 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:12.131094 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:12.131097 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:12.131589 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:12.131597 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:12.132038 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:12.132042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:12.132128 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:12.132480 (+ 352us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:07:12.132492 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:07:12.133886 (+ 1394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:07:12.136342 (+ 2456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:12.136346 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:12.136348 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:12.136569 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:12.136902 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:12.136905 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:12.136907 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:12.136993 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:12.137460 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:07:12.137463 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:07:12.137555 (+ 92us) client_negotiation.cc:769] Sending connection context
0808 06:07:12.137645 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 06:07:12.137770 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99}
Aug 08 06:07:12 dist-test-slave-zj8q krb5kdc[22707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169231, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.189@KRBTEST.COM for kudu/127.1.151.188@KRBTEST.COM
I20260808 06:07:12.138406 23074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:12.128451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:46179 (local address 127.0.0.1:45330)
0808 06:07:12.128551 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:12.128564 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:12.128671 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:12.128989 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:12.128992 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:12.129140 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:07:12.129222 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:12.129228 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:12.130517 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:12.130520 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:12.131110 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:12.131119 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:12.131730 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:12.131734 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:12.131805 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:12.132329 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:07:12.132342 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:07:12.134465 (+ 2123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:07:12.136944 (+ 2479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:12.136946 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:12.136948 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:12.137194 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:12.137467 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:12.137470 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:12.137472 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:12.137543 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:12.137933 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:07:12.137935 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:07:12.138013 (+ 78us) client_negotiation.cc:769] Sending connection context
0808 06:07:12.138104 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 06:07:12.138256 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 06:07:12.138572 23075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:12.129114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45340 (local address 127.1.151.188:46179)
0808 06:07:12.129459 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:12.129468 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:12.129482 (+ 14us) server_negotiation.cc:407] Connection header received
0808 06:07:12.129516 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:12.129518 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:12.129558 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:12.130120 (+ 562us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:12.130292 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:12.130998 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:12.131691 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:12.131937 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:12.132032 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:12.134040 (+ 2008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:12.134044 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:12.134046 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:12.134070 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:12.136219 (+ 2149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:12.136707 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:12.136710 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:12.136712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:12.136797 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:12.137107 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:12.137110 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:12.137112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:12.137351 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:12.138227 (+ 876us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:12.138301 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 06:07:12.138416 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":128,"threads_started":1}
I20260808 06:07:12.138865 23073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:12.128654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45330 (local address 127.1.151.188:46179)
0808 06:07:12.128751 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:12.128754 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:12.128766 (+ 12us) server_negotiation.cc:407] Connection header received
0808 06:07:12.128798 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:12.128800 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:12.128836 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:12.128905 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:12.129339 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:12.130429 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:12.131222 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:12.131552 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:12.131637 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:12.134580 (+ 2943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:12.134583 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:12.134585 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:12.134606 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:12.136855 (+ 2249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:12.137290 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:12.137293 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:12.137295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:12.137372 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:12.137624 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:12.137627 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:12.137629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:12.137811 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:12.137900 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:12.138634 (+ 734us) server_negotiation.cc:299] Negotiation successful
0808 06:07:12.138733 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 06:07:12.139326 22966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:12.129691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58796 (local address 127.1.151.189:46301)
0808 06:07:12.129798 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:12.129802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:12.129957 (+ 155us) server_negotiation.cc:407] Connection header received
0808 06:07:12.130057 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:12.130060 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:12.130099 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:12.130192 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:12.130878 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:12.131604 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:12.132595 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:12.132926 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:12.133002 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:12.135674 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:12.135678 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:12.135681 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:12.135707 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:12.137241 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:12.137683 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:12.137687 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:12.137688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:12.137769 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:12.138066 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:12.138069 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:12.138071 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:12.138341 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:12.138509 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:12.139062 (+ 553us) server_negotiation.cc:299] Negotiation successful
0808 06:07:12.139180 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 06:07:12.141727 23068 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6052fe3c96345cda502a5f6f74911a6: Bootstrap starting.
I20260808 06:07:12.141733 22824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e3e6a8eaa824d29a60a698db52f8c05: Bootstrap starting.
I20260808 06:07:12.143044 22958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:12.132518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:46179 (local address 127.0.0.1:45348)
0808 06:07:12.132743 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:12.132757 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:12.132859 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:12.133272 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:12.133275 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:12.133436 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:07:12.133527 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:12.133533 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:12.134946 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:12.134949 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:12.135612 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:12.135622 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:12.136823 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:12.136827 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:12.136914 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:12.137317 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:07:12.137331 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:07:12.139162 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:07:12.141464 (+ 2302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:12.141467 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:12.141469 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:12.141697 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:12.142035 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:12.142038 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:12.142040 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:12.142140 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:12.142599 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:07:12.142602 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:07:12.142692 (+ 90us) client_negotiation.cc:769] Sending connection context
0808 06:07:12.142771 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 06:07:12.142897 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 06:07:12.143644 23078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:12.132728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45348 (local address 127.1.151.188:46179)
0808 06:07:12.133046 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:12.133054 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:12.133066 (+ 12us) server_negotiation.cc:407] Connection header received
0808 06:07:12.133100 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:12.133103 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:12.133141 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:12.133698 (+ 557us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:12.133823 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:12.134845 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:12.135725 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:12.136682 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:12.139617 (+ 2935us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:12.139734 (+ 117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:12.139738 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:12.139740 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:12.139756 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:12.141345 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:12.141847 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:12.141850 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:12.141852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:12.141938 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:12.142276 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:12.142279 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:12.142281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:12.142486 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:12.143319 (+ 833us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:12.143392 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 06:07:12.143497 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":99,"threads_started":1}
I20260808 06:07:12.144315 22824 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e3e6a8eaa824d29a60a698db52f8c05: Neither blocks nor log segments found. Creating new log.
I20260808 06:07:12.144330 23068 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6052fe3c96345cda502a5f6f74911a6: Neither blocks nor log segments found. Creating new log.
I20260808 06:07:12.145021 22824 log.cc:824] T 00000000000000000000000000000000 P 7e3e6a8eaa824d29a60a698db52f8c05: Log is configured to *not* fsync() on all Append() calls
I20260808 06:07:12.145104 23068 log.cc:824] T 00000000000000000000000000000000 P f6052fe3c96345cda502a5f6f74911a6: Log is configured to *not* fsync() on all Append() calls
I20260808 06:07:12.146551 22955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c1f3d7ee0fd41329b59a8d18cc32452: Bootstrap starting.
I20260808 06:07:12.147290 22824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e3e6a8eaa824d29a60a698db52f8c05: No bootstrap required, opened a new log
I20260808 06:07:12.147303 23068 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6052fe3c96345cda502a5f6f74911a6: No bootstrap required, opened a new log
I20260808 06:07:12.148555 1630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 06:07:12.149039 22955 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c1f3d7ee0fd41329b59a8d18cc32452: Neither blocks nor log segments found. Creating new log.
I20260808 06:07:12.149824 22955 log.cc:824] T 00000000000000000000000000000000 P 1c1f3d7ee0fd41329b59a8d18cc32452: Log is configured to *not* fsync() on all Append() calls
I20260808 06:07:12.151288 22824 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e3e6a8eaa824d29a60a698db52f8c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e3e6a8eaa824d29a60a698db52f8c05" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39967 } } peers { permanent_uuid: "1c1f3d7ee0fd41329b59a8d18cc32452" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 46301 } } peers { permanent_uuid: "f6052fe3c96345cda502a5f6f74911a6" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 46179 } }
I20260808 06:07:12.151476 22824 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e3e6a8eaa824d29a60a698db52f8c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:07:12.151531 22824 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e3e6a8eaa824d29a60a698db52f8c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e3e6a8eaa824d29a60a698db52f8c05, State: Initialized, Role: FOLLOWER
I20260808 06:07:12.151409 23068 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6052fe3c96345cda502a5f6f74911a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e3e6a8eaa824d29a60a698db52f8c05" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39967 } } peers { permanent_uuid: "1c1f3d7ee0fd41329b59a8d18cc32452" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 46301 } } peers { permanent_uuid: "f6052fe3c96345cda502a5f6f74911a6" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 46179 } }
I20260808 06:07:12.151593 23068 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6052fe3c96345cda502a5f6f74911a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:07:12.151659 23068 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6052fe3c96345cda502a5f6f74911a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6052fe3c96345cda502a5f6f74911a6, State: Initialized, Role: FOLLOWER
I20260808 06:07:12.151947 22824 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e3e6a8eaa824d29a60a698db52f8c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e3e6a8eaa824d29a60a698db52f8c05" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39967 } } peers { permanent_uuid: "1c1f3d7ee0fd41329b59a8d18cc32452" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 46301 } } peers { permanent_uuid: "f6052fe3c96345cda502a5f6f74911a6" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 46179 } }
I20260808 06:07:12.152035 23068 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6052fe3c96345cda502a5f6f74911a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e3e6a8eaa824d29a60a698db52f8c05" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39967 } } peers { permanent_uuid: "1c1f3d7ee0fd41329b59a8d18cc32452" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 46301 } } peers { permanent_uuid: "f6052fe3c96345cda502a5f6f74911a6" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 46179 } }
I20260808 06:07:12.152354 23094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e3e6a8eaa824d29a60a698db52f8c05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e3e6a8eaa824d29a60a698db52f8c05" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39967 } } peers { permanent_uuid: "1c1f3d7ee0fd41329b59a8d18cc32452" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 46301 } } peers { permanent_uuid: "f6052fe3c96345cda502a5f6f74911a6" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 46179 } } }
I20260808 06:07:12.152456 23096 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6052fe3c96345cda502a5f6f74911a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e3e6a8eaa824d29a60a698db52f8c05" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39967 } } peers { permanent_uuid: "1c1f3d7ee0fd41329b59a8d18cc32452" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 46301 } } peers { permanent_uuid: "f6052fe3c96345cda502a5f6f74911a6" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 46179 } } }
I20260808 06:07:12.152583 23096 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6052fe3c96345cda502a5f6f74911a6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 06:07:12.152542 23094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e3e6a8eaa824d29a60a698db52f8c05 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 06:07:12.153110 22824 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e3e6a8eaa824d29a60a698db52f8c05 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:07:12.153110 23068 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6052fe3c96345cda502a5f6f74911a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:07:12.156024 22955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c1f3d7ee0fd41329b59a8d18cc32452: No bootstrap required, opened a new log
W20260808 06:07:12.159372 23134 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f6052fe3c96345cda502a5f6f74911a6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 06:07:12.159489 23134 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 06:07:12.160017 22955 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c1f3d7ee0fd41329b59a8d18cc32452 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e3e6a8eaa824d29a60a698db52f8c05" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39967 } } peers { permanent_uuid: "1c1f3d7ee0fd41329b59a8d18cc32452" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 46301 } } peers { permanent_uuid: "f6052fe3c96345cda502a5f6f74911a6" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 46179 } }
I20260808 06:07:12.160241 22955 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c1f3d7ee0fd41329b59a8d18cc32452 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:07:12.160296 22955 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c1f3d7ee0fd41329b59a8d18cc32452 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c1f3d7ee0fd41329b59a8d18cc32452, State: Initialized, Role: FOLLOWER
I20260808 06:07:12.160746 22955 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c1f3d7ee0fd41329b59a8d18cc32452 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e3e6a8eaa824d29a60a698db52f8c05" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39967 } } peers { permanent_uuid: "1c1f3d7ee0fd41329b59a8d18cc32452" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 46301 } } peers { permanent_uuid: "f6052fe3c96345cda502a5f6f74911a6" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 46179 } }
I20260808 06:07:12.157963 22724 master_runner.cc:186] Error getting master registration for 127.1.151.189:46301: OK, instance_id { permanent_uuid: "1c1f3d7ee0fd41329b59a8d18cc32452" instance_seqno: 1786169231879810 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 06:07:12.161774 22724 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 06:07:12.162057 22955 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c1f3d7ee0fd41329b59a8d18cc32452 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 06:07:12.164947 22754 tablet.cc:2369] T 00000000000000000000000000000000 P 7e3e6a8eaa824d29a60a698db52f8c05: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 06:07:12.167435 23125 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7e3e6a8eaa824d29a60a698db52f8c05: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 06:07:12.167510 23125 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 06:07:12.165385 22969 master_runner.cc:186] Error getting master registration for 127.1.151.189:46301: OK, instance_id { permanent_uuid: "1c1f3d7ee0fd41329b59a8d18cc32452" instance_seqno: 1786169231879810 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 06:07:12.168740 22969 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 06:07:12.169958 22828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:12.154531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52464 (local address 127.1.151.190:39967)
0808 06:07:12.154642 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:12.154646 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:12.156080 (+ 1434us) server_negotiation.cc:407] Connection header received
0808 06:07:12.156298 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:12.156301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:12.156354 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:12.156436 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:12.157363 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:12.158096 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:12.159322 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:12.159638 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:12.159716 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:12.161542 (+ 1826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:12.161546 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:12.161548 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:12.161574 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:12.163212 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:12.163717 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:12.163722 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:12.163724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:12.163817 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:12.164152 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:12.164156 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:12.164158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:12.164394 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:12.164519 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:12.169674 (+ 5155us) server_negotiation.cc:299] Negotiation successful
0808 06:07:12.169804 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 06:07:12.170733 22833 master_runner.cc:263] No leader master found from master 1c1f3d7ee0fd41329b59a8d18cc32452
I20260808 06:07:12.170795 22833 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 06:07:12.175472 23078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:12.154979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45350 (local address 127.1.151.188:46179)
0808 06:07:12.155071 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:12.155074 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:12.164638 (+ 9564us) server_negotiation.cc:407] Connection header received
0808 06:07:12.164756 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:12.164759 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:12.164810 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:12.164887 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:12.165689 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:12.166435 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:12.167480 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:12.167850 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:12.167927 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:12.171917 (+ 3990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:12.171921 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:12.171924 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:12.171952 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:12.173508 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:12.174003 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:12.174006 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:12.174008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:12.174097 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:12.174443 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:12.174446 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:12.174448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:12.174677 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:12.174794 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:12.175195 (+ 401us) server_negotiation.cc:299] Negotiation successful
0808 06:07:12.175319 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 06:07:12.176124 23139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c1f3d7ee0fd41329b59a8d18cc32452 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e3e6a8eaa824d29a60a698db52f8c05" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39967 } } peers { permanent_uuid: "1c1f3d7ee0fd41329b59a8d18cc32452" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 46301 } } peers { permanent_uuid: "f6052fe3c96345cda502a5f6f74911a6" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 46179 } } }
I20260808 06:07:12.176266 23139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c1f3d7ee0fd41329b59a8d18cc32452 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 06:07:12.176676 22966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:12.155400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58808 (local address 127.1.151.189:46301)
0808 06:07:12.155497 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:12.155501 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:12.167908 (+ 12407us) server_negotiation.cc:407] Connection header received
0808 06:07:12.168072 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:12.168076 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:12.168119 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:12.168165 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:12.168949 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:12.169625 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:12.170675 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:12.170979 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:12.171059 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:12.173098 (+ 2039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:12.173102 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:12.173104 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:12.173131 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:12.174747 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:12.175275 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:12.175279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:12.175281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:12.175374 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:12.175713 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:12.175717 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:12.175718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:12.175938 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:12.176037 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:12.176400 (+ 363us) server_negotiation.cc:299] Negotiation successful
0808 06:07:12.176523 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260808 06:07:12.185145 23153 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1c1f3d7ee0fd41329b59a8d18cc32452: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 06:07:12.185240 23153 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 06:07:12.234573 23139 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c1f3d7ee0fd41329b59a8d18cc32452 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:07:12.234802 23139 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c1f3d7ee0fd41329b59a8d18cc32452 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e3e6a8eaa824d29a60a698db52f8c05" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39967 } } peers { permanent_uuid: "1c1f3d7ee0fd41329b59a8d18cc32452" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 46301 } } peers { permanent_uuid: "f6052fe3c96345cda502a5f6f74911a6" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 46179 } }
I20260808 06:07:12.235971 23139 leader_election.cc:288] T 00000000000000000000000000000000 P 1c1f3d7ee0fd41329b59a8d18cc32452 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e3e6a8eaa824d29a60a698db52f8c05 (127.1.151.190:39967), f6052fe3c96345cda502a5f6f74911a6 (127.1.151.188:46179)
I20260808 06:07:12.236945 22788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1c1f3d7ee0fd41329b59a8d18cc32452" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e3e6a8eaa824d29a60a698db52f8c05" is_pre_election: true
I20260808 06:07:12.237286 22788 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7e3e6a8eaa824d29a60a698db52f8c05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c1f3d7ee0fd41329b59a8d18cc32452 in term 0.
I20260808 06:07:12.237733 22858 leader_election.cc:302] T 00000000000000000000000000000000 P 1c1f3d7ee0fd41329b59a8d18cc32452 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c1f3d7ee0fd41329b59a8d18cc32452, 7e3e6a8eaa824d29a60a698db52f8c05; no voters:
I20260808 06:07:12.238415 23026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1c1f3d7ee0fd41329b59a8d18cc32452" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6052fe3c96345cda502a5f6f74911a6" is_pre_election: true
I20260808 06:07:12.238719 23026 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f6052fe3c96345cda502a5f6f74911a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c1f3d7ee0fd41329b59a8d18cc32452 in term 0.
I20260808 06:07:12.239297 23139 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c1f3d7ee0fd41329b59a8d18cc32452 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 06:07:12.239396 23139 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c1f3d7ee0fd41329b59a8d18cc32452 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 06:07:12.239449 23139 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c1f3d7ee0fd41329b59a8d18cc32452 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:07:12.240785 23139 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c1f3d7ee0fd41329b59a8d18cc32452 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e3e6a8eaa824d29a60a698db52f8c05" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39967 } } peers { permanent_uuid: "1c1f3d7ee0fd41329b59a8d18cc32452" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 46301 } } peers { permanent_uuid: "f6052fe3c96345cda502a5f6f74911a6" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 46179 } }
I20260808 06:07:12.241503 22788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1c1f3d7ee0fd41329b59a8d18cc32452" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e3e6a8eaa824d29a60a698db52f8c05"
I20260808 06:07:12.241649 22788 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e3e6a8eaa824d29a60a698db52f8c05 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:07:12.243458 22788 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7e3e6a8eaa824d29a60a698db52f8c05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c1f3d7ee0fd41329b59a8d18cc32452 in term 1.
I20260808 06:07:12.243793 22858 leader_election.cc:302] T 00000000000000000000000000000000 P 1c1f3d7ee0fd41329b59a8d18cc32452 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c1f3d7ee0fd41329b59a8d18cc32452, 7e3e6a8eaa824d29a60a698db52f8c05; no voters:
I20260808 06:07:12.244351 23026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1c1f3d7ee0fd41329b59a8d18cc32452" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6052fe3c96345cda502a5f6f74911a6"
I20260808 06:07:12.245508 23026 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6052fe3c96345cda502a5f6f74911a6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:07:12.247378 23026 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f6052fe3c96345cda502a5f6f74911a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c1f3d7ee0fd41329b59a8d18cc32452 in term 1.
I20260808 06:07:12.250936 23139 leader_election.cc:288] T 00000000000000000000000000000000 P 1c1f3d7ee0fd41329b59a8d18cc32452 [CANDIDATE]: Term 1 election: Requested vote from peers 7e3e6a8eaa824d29a60a698db52f8c05 (127.1.151.190:39967), f6052fe3c96345cda502a5f6f74911a6 (127.1.151.188:46179)
I20260808 06:07:12.251116 23139 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c1f3d7ee0fd41329b59a8d18cc32452 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:07:12.251200 23139 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c1f3d7ee0fd41329b59a8d18cc32452 [term 1 LEADER]: Becoming Leader. State: Replica: 1c1f3d7ee0fd41329b59a8d18cc32452, State: Running, Role: LEADER
I20260808 06:07:12.251538 23139 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c1f3d7ee0fd41329b59a8d18cc32452 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e3e6a8eaa824d29a60a698db52f8c05" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39967 } } peers { permanent_uuid: "1c1f3d7ee0fd41329b59a8d18cc32452" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 46301 } } peers { permanent_uuid: "f6052fe3c96345cda502a5f6f74911a6" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 46179 } }
I20260808 06:07:12.253566 23139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c1f3d7ee0fd41329b59a8d18cc32452 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c1f3d7ee0fd41329b59a8d18cc32452. Latest consensus state: current_term: 1 leader_uuid: "1c1f3d7ee0fd41329b59a8d18cc32452" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e3e6a8eaa824d29a60a698db52f8c05" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39967 } } peers { permanent_uuid: "1c1f3d7ee0fd41329b59a8d18cc32452" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 46301 } } peers { permanent_uuid: "f6052fe3c96345cda502a5f6f74911a6" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 46179 } } }
I20260808 06:07:12.253674 23139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c1f3d7ee0fd41329b59a8d18cc32452 [sys.catalog]: This master's current role is: LEADER
I20260808 06:07:12.254236 23158 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:07:12.255240 23158 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:07:12.262419 23026 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f6052fe3c96345cda502a5f6f74911a6 [term 1 FOLLOWER]: Refusing update from remote peer 1c1f3d7ee0fd41329b59a8d18cc32452: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:07:12.263149 23156 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1c1f3d7ee0fd41329b59a8d18cc32452 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6052fe3c96345cda502a5f6f74911a6" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 46179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:07:12.267153 23096 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6052fe3c96345cda502a5f6f74911a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c1f3d7ee0fd41329b59a8d18cc32452. Latest consensus state: current_term: 1 leader_uuid: "1c1f3d7ee0fd41329b59a8d18cc32452" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e3e6a8eaa824d29a60a698db52f8c05" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39967 } } peers { permanent_uuid: "1c1f3d7ee0fd41329b59a8d18cc32452" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 46301 } } peers { permanent_uuid: "f6052fe3c96345cda502a5f6f74911a6" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 46179 } } }
I20260808 06:07:12.267279 23096 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6052fe3c96345cda502a5f6f74911a6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 06:07:12.270807 22788 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7e3e6a8eaa824d29a60a698db52f8c05 [term 1 FOLLOWER]: Refusing update from remote peer 1c1f3d7ee0fd41329b59a8d18cc32452: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:07:12.271445 23156 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1c1f3d7ee0fd41329b59a8d18cc32452 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e3e6a8eaa824d29a60a698db52f8c05" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:07:12.275527 23094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e3e6a8eaa824d29a60a698db52f8c05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c1f3d7ee0fd41329b59a8d18cc32452. Latest consensus state: current_term: 1 leader_uuid: "1c1f3d7ee0fd41329b59a8d18cc32452" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e3e6a8eaa824d29a60a698db52f8c05" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39967 } } peers { permanent_uuid: "1c1f3d7ee0fd41329b59a8d18cc32452" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 46301 } } peers { permanent_uuid: "f6052fe3c96345cda502a5f6f74911a6" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 46179 } } }
I20260808 06:07:12.275657 23094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e3e6a8eaa824d29a60a698db52f8c05 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 06:07:12.284605 23156 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c1f3d7ee0fd41329b59a8d18cc32452 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1c1f3d7ee0fd41329b59a8d18cc32452" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e3e6a8eaa824d29a60a698db52f8c05" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39967 } } peers { permanent_uuid: "1c1f3d7ee0fd41329b59a8d18cc32452" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 46301 } } peers { permanent_uuid: "f6052fe3c96345cda502a5f6f74911a6" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 46179 } } }
I20260808 06:07:12.284749 23156 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c1f3d7ee0fd41329b59a8d18cc32452 [sys.catalog]: This master's current role is: LEADER
I20260808 06:07:12.285970 23094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e3e6a8eaa824d29a60a698db52f8c05 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1c1f3d7ee0fd41329b59a8d18cc32452" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e3e6a8eaa824d29a60a698db52f8c05" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39967 } } peers { permanent_uuid: "1c1f3d7ee0fd41329b59a8d18cc32452" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 46301 } } peers { permanent_uuid: "f6052fe3c96345cda502a5f6f74911a6" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 46179 } } }
I20260808 06:07:12.286089 23094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e3e6a8eaa824d29a60a698db52f8c05 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 06:07:12.286499 23159 mvcc.cc:205] Tried to move back new op lower bound from 7316149175339094016 to 7316149175305576448. Current Snapshot: MvccSnapshot[applied={T|T < 7316149175339094016}]
I20260808 06:07:12.287447 23139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c1f3d7ee0fd41329b59a8d18cc32452 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1c1f3d7ee0fd41329b59a8d18cc32452" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e3e6a8eaa824d29a60a698db52f8c05" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39967 } } peers { permanent_uuid: "1c1f3d7ee0fd41329b59a8d18cc32452" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 46301 } } peers { permanent_uuid: "f6052fe3c96345cda502a5f6f74911a6" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 46179 } } }
I20260808 06:07:12.287575 23139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c1f3d7ee0fd41329b59a8d18cc32452 [sys.catalog]: This master's current role is: LEADER
I20260808 06:07:12.288475 23096 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6052fe3c96345cda502a5f6f74911a6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1c1f3d7ee0fd41329b59a8d18cc32452" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e3e6a8eaa824d29a60a698db52f8c05" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39967 } } peers { permanent_uuid: "1c1f3d7ee0fd41329b59a8d18cc32452" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 46301 } } peers { permanent_uuid: "f6052fe3c96345cda502a5f6f74911a6" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 46179 } } }
I20260808 06:07:12.288604 23096 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6052fe3c96345cda502a5f6f74911a6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 06:07:12.290237 23163 mvcc.cc:205] Tried to move back new op lower bound from 7316149175339094016 to 7316149175305576448. Current Snapshot: MvccSnapshot[applied={T|T < 7316149175339094016}]
I20260808 06:07:12.290419 23160 mvcc.cc:205] Tried to move back new op lower bound from 7316149175339094016 to 7316149175305576448. Current Snapshot: MvccSnapshot[applied={T|T < 7316149175339094016}]
I20260808 06:07:12.296423 23158 catalog_manager.cc:1347] Generated new cluster ID: 984f5ca1c5c546f1944d0901c40dc59a
I20260808 06:07:12.296500 23158 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:07:12.347362 23158 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:07:12.349905 23158 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:07:12.363154 23158 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c1f3d7ee0fd41329b59a8d18cc32452: Generated new TSK 0
I20260808 06:07:12.365748 23158 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 06:07:13.160946 23134 catalog_manager.cc:1259] Loaded cluster ID: 984f5ca1c5c546f1944d0901c40dc59a
I20260808 06:07:13.161029 23134 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f6052fe3c96345cda502a5f6f74911a6: loading cluster ID for follower catalog manager: success
I20260808 06:07:13.162822 23134 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f6052fe3c96345cda502a5f6f74911a6: acquiring CA information for follower catalog manager: success
I20260808 06:07:13.164084 23134 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f6052fe3c96345cda502a5f6f74911a6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 06:07:13.168593 23125 catalog_manager.cc:1259] Loaded cluster ID: 984f5ca1c5c546f1944d0901c40dc59a
I20260808 06:07:13.168666 23125 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7e3e6a8eaa824d29a60a698db52f8c05: loading cluster ID for follower catalog manager: success
I20260808 06:07:13.170209 23125 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7e3e6a8eaa824d29a60a698db52f8c05: acquiring CA information for follower catalog manager: success
I20260808 06:07:13.171267 23125 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7e3e6a8eaa824d29a60a698db52f8c05: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 06:07:13.435190 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 22724
I20260808 06:07:13.445255 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 22833
I20260808 06:07:13.458102 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 22969
2026-08-08T06:07:13Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3236 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:07:13 dist-test-slave-zj8q krb5kdc[23178](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:07:13 dist-test-slave-zj8q krb5kdc[23178](info): set up 2 sockets
Aug 08 06:07:13 dist-test-slave-zj8q krb5kdc[23178](info): commencing operation
krb5kdc: starting...
W20260808 06:07:14.596928 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:07:14 dist-test-slave-zj8q krb5kdc[23178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169234, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:07:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:07:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.151.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
I20260808 06:07:14.714511 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:36953
--webserver_interface=127.1.151.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:42025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:36953
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:07:14.822994 23199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:07:14.823242 23199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:07:14.823283 23199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:07:14.827211 23199 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:07:14.827261 23199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:07:14.827280 23199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:07:14.827298 23199 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:07:14.827327 23199 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:07:14.832368 23199 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:42025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:36953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:36953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:07:14.833309 23199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:07:14.834295 23199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:07:14.842384 23206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:14.842378 23210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:14.842358 23199 server_base.cc:1034] running on GCE node
W20260808 06:07:14.842353 23207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:14.843134 23199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:07:14.844164 23199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:07:14.845424 23199 hybrid_clock.cc:648] HybridClock initialized: now 1786169234845407 us; error 25 us; skew 500 ppm
Aug 08 06:07:14 dist-test-slave-zj8q krb5kdc[23178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169234, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:07:14.848201 23199 init.cc:377] Logged in from keytab as kudu/127.1.151.190@KRBTEST.COM (short username kudu)
I20260808 06:07:14.849077 23199 webserver.cc:460] Webserver started at http://127.1.151.190:36941/ using document root <none> and password file <none>
I20260808 06:07:14.849633 23199 fs_manager.cc:359] Metadata directory not provided
I20260808 06:07:14.849704 23199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:07:14.849900 23199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:07:14.851926 23199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "bde0102e7fcc4c71a31506d655ae485f"
format_stamp: "Formatted at 2026-08-08 06:07:14 on dist-test-slave-zj8q"
server_key: "94f9fe83d512bb0ab6d5e56935ae2aa8"
server_key_iv: "436a298e464ba20d4a5f7110c68c2629"
server_key_version: "encryptionkey@0"
I20260808 06:07:14.852469 23199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "bde0102e7fcc4c71a31506d655ae485f"
format_stamp: "Formatted at 2026-08-08 06:07:14 on dist-test-slave-zj8q"
server_key: "94f9fe83d512bb0ab6d5e56935ae2aa8"
server_key_iv: "436a298e464ba20d4a5f7110c68c2629"
server_key_version: "encryptionkey@0"
I20260808 06:07:14.856115 23199 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 06:07:14.858414 23222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:14.859571 23199 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 06:07:14.859683 23199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "bde0102e7fcc4c71a31506d655ae485f"
format_stamp: "Formatted at 2026-08-08 06:07:14 on dist-test-slave-zj8q"
server_key: "94f9fe83d512bb0ab6d5e56935ae2aa8"
server_key_iv: "436a298e464ba20d4a5f7110c68c2629"
server_key_version: "encryptionkey@0"
I20260808 06:07:14.859771 23199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:07:14.884920 23199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:07:14.887974 23199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:07:14.888147 23199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:07:14.895071 23199 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:36953
I20260808 06:07:14.895098 23307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:36953 every 8 connection(s)
I20260808 06:07:14.895988 23199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:07:14.898667 23308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:07:14.900019 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 23199
I20260808 06:07:14.900157 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:07:14.900480 1630 external_mini_cluster.cc:1428] Setting key bed3d4a9ff3891209cffcf431f840082
I20260808 06:07:14.905296 23308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bde0102e7fcc4c71a31506d655ae485f: Bootstrap starting.
I20260808 06:07:14.907908 23308 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bde0102e7fcc4c71a31506d655ae485f: Neither blocks nor log segments found. Creating new log.
I20260808 06:07:14.908677 23308 log.cc:824] T 00000000000000000000000000000000 P bde0102e7fcc4c71a31506d655ae485f: Log is configured to *not* fsync() on all Append() calls
Aug 08 06:07:14 dist-test-slave-zj8q krb5kdc[23178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169234, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:07:14.910777 23308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bde0102e7fcc4c71a31506d655ae485f: No bootstrap required, opened a new log
I20260808 06:07:14.914813 23316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:14.901794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40252 (local address 127.1.151.190:36953)
0808 06:07:14.902230 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:14.902250 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:14.902292 (+ 42us) server_negotiation.cc:407] Connection header received
0808 06:07:14.903096 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:14.903120 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:14.903520 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:14.903879 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:14.904522 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:14.905987 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:14.906904 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:14.907573 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:14.907738 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:14.910245 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:14.910250 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:14.910263 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:14.910289 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:14.912158 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:14.912627 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:14.912631 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:14.912635 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:14.912737 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:14.913004 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:14.913006 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:14.913007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:14.913371 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:14.913516 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:14.913794 (+ 278us) server_negotiation.cc:299] Negotiation successful
0808 06:07:14.914051 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":105,"threads_started":1}
I20260808 06:07:14.914840 23308 raft_consensus.cc:348] T 00000000000000000000000000000000 P bde0102e7fcc4c71a31506d655ae485f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bde0102e7fcc4c71a31506d655ae485f" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 36953 } }
I20260808 06:07:14.915109 23308 raft_consensus.cc:374] T 00000000000000000000000000000000 P bde0102e7fcc4c71a31506d655ae485f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:07:14.915166 23308 raft_consensus.cc:729] T 00000000000000000000000000000000 P bde0102e7fcc4c71a31506d655ae485f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bde0102e7fcc4c71a31506d655ae485f, State: Initialized, Role: FOLLOWER
I20260808 06:07:14.915609 23308 consensus_queue.cc:260] T 00000000000000000000000000000000 P bde0102e7fcc4c71a31506d655ae485f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bde0102e7fcc4c71a31506d655ae485f" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 36953 } }
I20260808 06:07:14.915750 23308 raft_consensus.cc:388] T 00000000000000000000000000000000 P bde0102e7fcc4c71a31506d655ae485f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:07:14.915813 23308 raft_consensus.cc:482] T 00000000000000000000000000000000 P bde0102e7fcc4c71a31506d655ae485f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:07:14.915913 23308 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bde0102e7fcc4c71a31506d655ae485f [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:07:14.916801 23308 raft_consensus.cc:504] T 00000000000000000000000000000000 P bde0102e7fcc4c71a31506d655ae485f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bde0102e7fcc4c71a31506d655ae485f" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 36953 } }
I20260808 06:07:14.917156 23308 leader_election.cc:302] T 00000000000000000000000000000000 P bde0102e7fcc4c71a31506d655ae485f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bde0102e7fcc4c71a31506d655ae485f; no voters:
I20260808 06:07:14.917474 23308 leader_election.cc:288] T 00000000000000000000000000000000 P bde0102e7fcc4c71a31506d655ae485f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:07:14.917589 1630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 06:07:14.917580 23319 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bde0102e7fcc4c71a31506d655ae485f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:07:14.917790 23319 raft_consensus.cc:686] T 00000000000000000000000000000000 P bde0102e7fcc4c71a31506d655ae485f [term 1 LEADER]: Becoming Leader. State: Replica: bde0102e7fcc4c71a31506d655ae485f, State: Running, Role: LEADER
I20260808 06:07:14.918064 23319 consensus_queue.cc:237] T 00000000000000000000000000000000 P bde0102e7fcc4c71a31506d655ae485f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bde0102e7fcc4c71a31506d655ae485f" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 36953 } }
I20260808 06:07:14.918725 23308 sys_catalog.cc:564] T 00000000000000000000000000000000 P bde0102e7fcc4c71a31506d655ae485f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:07:14.919682 23321 sys_catalog.cc:455] T 00000000000000000000000000000000 P bde0102e7fcc4c71a31506d655ae485f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bde0102e7fcc4c71a31506d655ae485f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bde0102e7fcc4c71a31506d655ae485f" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 36953 } } }
I20260808 06:07:14.919813 23321 sys_catalog.cc:458] T 00000000000000000000000000000000 P bde0102e7fcc4c71a31506d655ae485f [sys.catalog]: This master's current role is: LEADER
I20260808 06:07:14.920035 23320 sys_catalog.cc:455] T 00000000000000000000000000000000 P bde0102e7fcc4c71a31506d655ae485f [sys.catalog]: SysCatalogTable state changed. Reason: New leader bde0102e7fcc4c71a31506d655ae485f. Latest consensus state: current_term: 1 leader_uuid: "bde0102e7fcc4c71a31506d655ae485f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bde0102e7fcc4c71a31506d655ae485f" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 36953 } } }
I20260808 06:07:14.920137 23320 sys_catalog.cc:458] T 00000000000000000000000000000000 P bde0102e7fcc4c71a31506d655ae485f [sys.catalog]: This master's current role is: LEADER
I20260808 06:07:14.920267 23336 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:07:14.922842 23336 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:07:14.927830 23336 catalog_manager.cc:1347] Generated new cluster ID: 510031fb3df94b30ab7d9c22ff50fe68
I20260808 06:07:14.927882 23336 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:07:14.933960 23336 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:07:14.934804 23336 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:07:14.946012 23336 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bde0102e7fcc4c71a31506d655ae485f: Generated new TSK 0
I20260808 06:07:14.946662 23336 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 06:07:14.973335 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 23199
2026-08-08T06:07:14Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1516 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:07:15 dist-test-slave-zj8q krb5kdc[23362](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:07:15 dist-test-slave-zj8q krb5kdc[23362](info): set up 2 sockets
Aug 08 06:07:15 dist-test-slave-zj8q krb5kdc[23362](info): commencing operation
krb5kdc: starting...
W20260808 06:07:16.117763 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:07:16 dist-test-slave-zj8q krb5kdc[23362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:07:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:07:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.151.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
I20260808 06:07:16.240370 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:39611
--webserver_interface=127.1.151.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:34817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:39611
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:07:16.350867 23383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:07:16.351122 23383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:07:16.351179 23383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:07:16.355281 23383 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:07:16.355335 23383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:07:16.355360 23383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:07:16.355384 23383 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:07:16.355412 23383 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:07:16.360525 23383 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:34817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:39611
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:39611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:07:16.361483 23383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:07:16.362457 23383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:07:16.370548 23392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:16.370554 23396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:16.370508 23391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:16.370975 23383 server_base.cc:1034] running on GCE node
I20260808 06:07:16.371505 23383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:07:16.372552 23383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:07:16.373730 23383 hybrid_clock.cc:648] HybridClock initialized: now 1786169236373716 us; error 26 us; skew 500 ppm
Aug 08 06:07:16 dist-test-slave-zj8q krb5kdc[23362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169236, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:07:16.380477 23383 init.cc:377] Logged in from keytab as kudu/127.1.151.190@KRBTEST.COM (short username kudu)
I20260808 06:07:16.381470 23383 webserver.cc:460] Webserver started at http://127.1.151.190:41435/ using document root <none> and password file <none>
I20260808 06:07:16.382064 23383 fs_manager.cc:359] Metadata directory not provided
I20260808 06:07:16.382138 23383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:07:16.382354 23383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:07:16.384366 23383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "51bb59562e294210867a4e6920f254b6"
format_stamp: "Formatted at 2026-08-08 06:07:16 on dist-test-slave-zj8q"
server_key: "040f1b8315b7842701fb291a6e279abb"
server_key_iv: "5a2b5e974e7e7606478bb0600ab4c863"
server_key_version: "encryptionkey@0"
I20260808 06:07:16.384914 23383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "51bb59562e294210867a4e6920f254b6"
format_stamp: "Formatted at 2026-08-08 06:07:16 on dist-test-slave-zj8q"
server_key: "040f1b8315b7842701fb291a6e279abb"
server_key_iv: "5a2b5e974e7e7606478bb0600ab4c863"
server_key_version: "encryptionkey@0"
I20260808 06:07:16.388624 23383 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 06:07:16.390981 23409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:16.392153 23383 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:07:16.392277 23383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "51bb59562e294210867a4e6920f254b6"
format_stamp: "Formatted at 2026-08-08 06:07:16 on dist-test-slave-zj8q"
server_key: "040f1b8315b7842701fb291a6e279abb"
server_key_iv: "5a2b5e974e7e7606478bb0600ab4c863"
server_key_version: "encryptionkey@0"
I20260808 06:07:16.392364 23383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:07:16.436663 23383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:07:16.439970 23383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:07:16.440155 23383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:07:16.447001 23383 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:39611
I20260808 06:07:16.447260 23486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:39611 every 8 connection(s)
I20260808 06:07:16.447949 23383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:07:16.450659 23488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:07:16.456197 23488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6: Bootstrap starting.
I20260808 06:07:16.456974 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 23383
I20260808 06:07:16.457115 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:07:16.457437 1630 external_mini_cluster.cc:1428] Setting key 2e2531a93f9dae0d2bd10330440db091
I20260808 06:07:16.459230 23488 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6: Neither blocks nor log segments found. Creating new log.
I20260808 06:07:16.459972 23488 log.cc:824] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6: Log is configured to *not* fsync() on all Append() calls
I20260808 06:07:16.462123 23488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6: No bootstrap required, opened a new log
Aug 08 06:07:16 dist-test-slave-zj8q krb5kdc[23362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:07:16.466580 23488 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51bb59562e294210867a4e6920f254b6" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39611 } }
I20260808 06:07:16.466813 23488 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:07:16.466879 23488 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51bb59562e294210867a4e6920f254b6, State: Initialized, Role: FOLLOWER
I20260808 06:07:16.467317 23488 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51bb59562e294210867a4e6920f254b6" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39611 } }
I20260808 06:07:16.467454 23488 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:07:16.467519 23488 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:07:16.467614 23488 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:07:16.468449 23488 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51bb59562e294210867a4e6920f254b6" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39611 } }
I20260808 06:07:16.468775 23488 leader_election.cc:302] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51bb59562e294210867a4e6920f254b6; no voters:
I20260808 06:07:16.469058 23488 leader_election.cc:288] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:07:16.469146 23493 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:07:16.469355 23493 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [term 1 LEADER]: Becoming Leader. State: Replica: 51bb59562e294210867a4e6920f254b6, State: Running, Role: LEADER
I20260808 06:07:16.469748 23493 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51bb59562e294210867a4e6920f254b6" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39611 } }
I20260808 06:07:16.470300 23488 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:07:16.470960 23496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51bb59562e294210867a4e6920f254b6. Latest consensus state: current_term: 1 leader_uuid: "51bb59562e294210867a4e6920f254b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51bb59562e294210867a4e6920f254b6" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39611 } } }
I20260808 06:07:16.471100 23496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [sys.catalog]: This master's current role is: LEADER
I20260808 06:07:16.471757 23494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "51bb59562e294210867a4e6920f254b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51bb59562e294210867a4e6920f254b6" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39611 } } }
I20260808 06:07:16.471930 23494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [sys.catalog]: This master's current role is: LEADER
I20260808 06:07:16.473237 23491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:16.458754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36380 (local address 127.1.151.190:39611)
0808 06:07:16.459180 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:16.459196 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:16.459232 (+ 36us) server_negotiation.cc:407] Connection header received
0808 06:07:16.459883 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:16.459900 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:16.460254 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:16.460620 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:16.461446 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:16.462525 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:16.463743 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:16.464234 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:16.464357 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:16.467181 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:16.467186 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:16.467199 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:16.467225 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:16.468925 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:16.469442 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:16.469449 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:16.469456 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:16.469597 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:16.469982 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:16.469985 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:16.469987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:16.470550 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:16.470779 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:16.471593 (+ 814us) server_negotiation.cc:299] Negotiation successful
0808 06:07:16.471896 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":125,"threads_started":1}
W20260808 06:07:16.475514 23517 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 06:07:16.475625 23517 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 06:07:16.475771 23506 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:07:16.476482 23506 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:07:16.481815 23506 catalog_manager.cc:1347] Generated new cluster ID: 7bdd4ae20a4e4ac18482b0cdadc08061
I20260808 06:07:16.481881 23506 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:07:16.497386 23506 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:07:16.498322 23506 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:07:16.508769 23506 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6: Generated new TSK 0
I20260808 06:07:16.509407 23506 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.151.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab.
Entry for principal kudu/127.1.151.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab.
I20260808 06:07:16.537385 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.129:0
--local_ip_for_outbound_sockets=127.1.151.129
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:39611
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:34817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:07:16.648687 23525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:07:16.648922 23525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:07:16.648962 23525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:07:16.652892 23525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:07:16.652949 23525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:07:16.653025 23525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.129
I20260808 06:07:16.658013 23525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:34817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:39611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.129
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:07:16.658988 23525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:07:16.659960 23525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:07:16.669191 23532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:16.669410 23525 server_base.cc:1034] running on GCE node
W20260808 06:07:16.669446 23531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:16.669791 23537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:16.670274 23525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:07:16.671010 23525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:07:16.672227 23525 hybrid_clock.cc:648] HybridClock initialized: now 1786169236672155 us; error 85 us; skew 500 ppm
Aug 08 06:07:16 dist-test-slave-zj8q krb5kdc[23362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169236, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:07:16.675635 23525 init.cc:377] Logged in from keytab as kudu/127.1.151.129@KRBTEST.COM (short username kudu)
I20260808 06:07:16.678535 23525 webserver.cc:460] Webserver started at http://127.1.151.129:32961/ using document root <none> and password file <none>
I20260808 06:07:16.679205 23525 fs_manager.cc:359] Metadata directory not provided
I20260808 06:07:16.679279 23525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:07:16.679482 23525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:07:16.682013 23525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/data/instance:
uuid: "7ce698571e9f4da3a09c77e6ad265cd2"
format_stamp: "Formatted at 2026-08-08 06:07:16 on dist-test-slave-zj8q"
server_key: "dbff06773774fe574da4b5b669a147ba"
server_key_iv: "ffba471894e68b82de61319530746781"
server_key_version: "encryptionkey@0"
I20260808 06:07:16.682729 23525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance:
uuid: "7ce698571e9f4da3a09c77e6ad265cd2"
format_stamp: "Formatted at 2026-08-08 06:07:16 on dist-test-slave-zj8q"
server_key: "dbff06773774fe574da4b5b669a147ba"
server_key_iv: "ffba471894e68b82de61319530746781"
server_key_version: "encryptionkey@0"
I20260808 06:07:16.690582 23525 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.000s
I20260808 06:07:16.693511 23547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:16.694880 23525 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 06:07:16.694995 23525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/wal
uuid: "7ce698571e9f4da3a09c77e6ad265cd2"
format_stamp: "Formatted at 2026-08-08 06:07:16 on dist-test-slave-zj8q"
server_key: "dbff06773774fe574da4b5b669a147ba"
server_key_iv: "ffba471894e68b82de61319530746781"
server_key_version: "encryptionkey@0"
I20260808 06:07:16.695120 23525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:07:16.711320 23525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:07:16.714692 23525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:07:16.714874 23525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:07:16.715487 23525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:07:16.716305 23525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:07:16.716352 23525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:16.716394 23525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:07:16.716421 23525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:16.725620 23525 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.129:39351
I20260808 06:07:16.725674 23733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.129:39351 every 8 connection(s)
I20260808 06:07:16.726548 23525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
I20260808 06:07:16.734388 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 23525
I20260808 06:07:16.734516 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance
I20260808 06:07:16.734793 1630 external_mini_cluster.cc:1428] Setting key f1d52c5d1d5ed47d678e9f9c438b6d90
Aug 08 06:07:16 dist-test-slave-zj8q krb5kdc[23362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169236, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:07:16.739337 23491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:16.728284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:47929 (local address 127.1.151.190:39611)
0808 06:07:16.728463 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:16.728468 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:16.729465 (+ 997us) server_negotiation.cc:407] Connection header received
0808 06:07:16.730401 (+ 936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:16.730404 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:16.730451 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:16.730523 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:16.731614 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:16.732142 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:16.733235 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:16.733462 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:16.733527 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:16.735956 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:16.735959 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:16.735961 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:16.735984 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:16.737285 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:16.737830 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:16.737832 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:16.737834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:16.737906 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:16.738262 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:16.738266 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:16.738268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:16.738491 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:16.738584 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:16.739049 (+ 465us) server_negotiation.cc:299] Negotiation successful
0808 06:07:16.739175 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 06:07:16.740111 23737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:16.728677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.129:47929)
0808 06:07:16.729330 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:16.729369 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:16.730110 (+ 741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:16.730699 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:16.730706 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:16.731047 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:07:16.731494 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:16.731507 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:16.732231 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:16.732233 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:16.733136 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:16.733147 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:16.733587 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:16.733590 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:16.733761 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:16.734281 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:07:16.734297 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:07:16.735800 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:07:16.737398 (+ 1598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:16.737403 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:16.737417 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:16.737745 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:16.737980 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:16.737982 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:16.737984 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:16.738175 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:16.738591 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:07:16.738596 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:07:16.738826 (+ 230us) client_negotiation.cc:769] Sending connection context
0808 06:07:16.738964 (+ 138us) client_negotiation.cc:240] Negotiation successful
0808 06:07:16.739210 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":122,"threads_started":1}
I20260808 06:07:16.741271 23734 heartbeater.cc:347] Connected to a master server at 127.1.151.190:39611
I20260808 06:07:16.741534 23734 heartbeater.cc:464] Registering TS with master...
I20260808 06:07:16.742121 23734 heartbeater.cc:511] Master 127.1.151.190:39611 requested a full tablet report, sending...
I20260808 06:07:16.743692 23433 ts_manager.cc:194] Registered new tserver with Master: 7ce698571e9f4da3a09c77e6ad265cd2 (127.1.151.129:39351)
I20260808 06:07:16.744844 23433 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.129@KRBTEST.COM'} at 127.1.151.129:47929
WARNING: no policy specified for kudu/127.1.151.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab.
Entry for principal kudu/127.1.151.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab.
I20260808 06:07:16.762373 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.130:0
--local_ip_for_outbound_sockets=127.1.151.130
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:39611
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:34817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:07:16.872771 23742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:07:16.873013 23742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:07:16.873071 23742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:07:16.876982 23742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:07:16.877035 23742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:07:16.877111 23742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.130
I20260808 06:07:16.882210 23742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:34817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:39611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.130
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:07:16.883200 23742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:07:16.884205 23742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:07:16.893415 23753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:16.893500 23742 server_base.cc:1034] running on GCE node
W20260808 06:07:16.893452 23751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:16.893441 23749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:16.894002 23742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:07:16.894649 23742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:07:16.895802 23742 hybrid_clock.cc:648] HybridClock initialized: now 1786169236895786 us; error 28 us; skew 500 ppm
Aug 08 06:07:16 dist-test-slave-zj8q krb5kdc[23362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169236, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:07:16.898551 23742 init.cc:377] Logged in from keytab as kudu/127.1.151.130@KRBTEST.COM (short username kudu)
I20260808 06:07:16.899488 23742 webserver.cc:460] Webserver started at http://127.1.151.130:33379/ using document root <none> and password file <none>
I20260808 06:07:16.900019 23742 fs_manager.cc:359] Metadata directory not provided
I20260808 06:07:16.900086 23742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:07:16.900277 23742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:07:16.902241 23742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/data/instance:
uuid: "38fc7605759a46d6b9e67bfb0d13372c"
format_stamp: "Formatted at 2026-08-08 06:07:16 on dist-test-slave-zj8q"
server_key: "47e94270c951564af4414842d69126ca"
server_key_iv: "d47dd0cb305710dbfdacea7d7112ae8f"
server_key_version: "encryptionkey@0"
I20260808 06:07:16.902791 23742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance:
uuid: "38fc7605759a46d6b9e67bfb0d13372c"
format_stamp: "Formatted at 2026-08-08 06:07:16 on dist-test-slave-zj8q"
server_key: "47e94270c951564af4414842d69126ca"
server_key_iv: "d47dd0cb305710dbfdacea7d7112ae8f"
server_key_version: "encryptionkey@0"
I20260808 06:07:16.906327 23742 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 06:07:16.908629 23763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:16.909752 23742 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:07:16.909859 23742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/wal
uuid: "38fc7605759a46d6b9e67bfb0d13372c"
format_stamp: "Formatted at 2026-08-08 06:07:16 on dist-test-slave-zj8q"
server_key: "47e94270c951564af4414842d69126ca"
server_key_iv: "d47dd0cb305710dbfdacea7d7112ae8f"
server_key_version: "encryptionkey@0"
I20260808 06:07:16.909948 23742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:07:16.927738 23742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:07:16.930441 23742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:07:16.930613 23742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:07:16.931180 23742 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:07:16.931993 23742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:07:16.932040 23742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:16.932094 23742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:07:16.932124 23742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:16.940285 23742 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.130:45793
I20260808 06:07:16.940330 23952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.130:45793 every 8 connection(s)
I20260808 06:07:16.941169 23742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
I20260808 06:07:16.948714 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 23742
I20260808 06:07:16.948835 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance
I20260808 06:07:16.949147 1630 external_mini_cluster.cc:1428] Setting key 6dc3685ae37b7c60de6b6268fcbb0ce0
Aug 08 06:07:16 dist-test-slave-zj8q krb5kdc[23362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169236, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:07:16.955001 23491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:16.943165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:48649 (local address 127.1.151.190:39611)
0808 06:07:16.943295 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:16.943299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:16.944324 (+ 1025us) server_negotiation.cc:407] Connection header received
0808 06:07:16.945254 (+ 930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:16.945258 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:16.945319 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:16.945395 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:16.946649 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:16.947185 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:16.948345 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:16.948580 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:16.948787 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:16.951664 (+ 2877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:16.951668 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:16.951670 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:16.951695 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:16.952986 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:16.953489 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:16.953493 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:16.953495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:16.953589 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:16.953888 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:16.953891 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:16.953893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:16.954118 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:16.954247 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:16.954729 (+ 482us) server_negotiation.cc:299] Negotiation successful
0808 06:07:16.954853 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 06:07:16.955765 23957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:16.943480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.130:48649)
0808 06:07:16.944177 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:16.944221 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:16.945042 (+ 821us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:16.945600 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:16.945607 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:16.945957 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:07:16.946490 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:16.946505 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:16.947308 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:16.947311 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:16.948216 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:16.948226 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:16.948796 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:16.948800 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:16.949006 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:16.949713 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:07:16.949734 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:07:16.951459 (+ 1725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:07:16.953117 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:16.953122 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:16.953133 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:16.953398 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:16.953673 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:16.953675 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:16.953676 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:16.953808 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:16.954243 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:07:16.954248 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:07:16.954477 (+ 229us) client_negotiation.cc:769] Sending connection context
0808 06:07:16.954641 (+ 164us) client_negotiation.cc:240] Negotiation successful
0808 06:07:16.954896 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":120,"threads_started":1}
I20260808 06:07:16.956977 23955 heartbeater.cc:347] Connected to a master server at 127.1.151.190:39611
I20260808 06:07:16.957201 23955 heartbeater.cc:464] Registering TS with master...
I20260808 06:07:16.957705 23955 heartbeater.cc:511] Master 127.1.151.190:39611 requested a full tablet report, sending...
I20260808 06:07:16.958854 23433 ts_manager.cc:194] Registered new tserver with Master: 38fc7605759a46d6b9e67bfb0d13372c (127.1.151.130:45793)
I20260808 06:07:16.959421 23433 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.130@KRBTEST.COM'} at 127.1.151.130:48649
WARNING: no policy specified for kudu/127.1.151.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab.
Entry for principal kudu/127.1.151.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab.
I20260808 06:07:16.976351 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.131:0
--local_ip_for_outbound_sockets=127.1.151.131
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:39611
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:34817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:07:17.087217 23965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:07:17.087512 23965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:07:17.087590 23965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:07:17.091579 23965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:07:17.091629 23965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:07:17.091696 23965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.131
I20260808 06:07:17.096825 23965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:34817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:39611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.131
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:07:17.097812 23965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:07:17.098845 23965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:07:17.108283 23974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:17.108283 23970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:17.108264 23971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:17.108455 23965 server_base.cc:1034] running on GCE node
I20260808 06:07:17.108898 23965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:07:17.109632 23965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:07:17.110795 23965 hybrid_clock.cc:648] HybridClock initialized: now 1786169237110774 us; error 30 us; skew 500 ppm
Aug 08 06:07:17 dist-test-slave-zj8q krb5kdc[23362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169237, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:07:17.113390 23965 init.cc:377] Logged in from keytab as kudu/127.1.151.131@KRBTEST.COM (short username kudu)
I20260808 06:07:17.114284 23965 webserver.cc:460] Webserver started at http://127.1.151.131:33361/ using document root <none> and password file <none>
I20260808 06:07:17.114833 23965 fs_manager.cc:359] Metadata directory not provided
I20260808 06:07:17.114894 23965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:07:17.115090 23965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:07:17.117060 23965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/data/instance:
uuid: "2bbf31860ef14a1c8b7fed6f6a415153"
format_stamp: "Formatted at 2026-08-08 06:07:17 on dist-test-slave-zj8q"
server_key: "8180099d00e94bae7912f5617beffd41"
server_key_iv: "cc4a4e57cb71a2be30cfc51900df39be"
server_key_version: "encryptionkey@0"
I20260808 06:07:17.117602 23965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance:
uuid: "2bbf31860ef14a1c8b7fed6f6a415153"
format_stamp: "Formatted at 2026-08-08 06:07:17 on dist-test-slave-zj8q"
server_key: "8180099d00e94bae7912f5617beffd41"
server_key_iv: "cc4a4e57cb71a2be30cfc51900df39be"
server_key_version: "encryptionkey@0"
I20260808 06:07:17.121073 23965 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260808 06:07:17.123406 23983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:17.124570 23965 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 06:07:17.124675 23965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/wal
uuid: "2bbf31860ef14a1c8b7fed6f6a415153"
format_stamp: "Formatted at 2026-08-08 06:07:17 on dist-test-slave-zj8q"
server_key: "8180099d00e94bae7912f5617beffd41"
server_key_iv: "cc4a4e57cb71a2be30cfc51900df39be"
server_key_version: "encryptionkey@0"
I20260808 06:07:17.124761 23965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:07:17.141608 23965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:07:17.144243 23965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:07:17.144402 23965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:07:17.144959 23965 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:07:17.145731 23965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:07:17.145777 23965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:17.145814 23965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:07:17.145834 23965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:17.154021 23965 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.131:43389
I20260808 06:07:17.154057 24161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.131:43389 every 8 connection(s)
I20260808 06:07:17.154853 23965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
I20260808 06:07:17.162552 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 23965
I20260808 06:07:17.162683 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance
I20260808 06:07:17.162995 1630 external_mini_cluster.cc:1428] Setting key abaa23b72ac361845338df4b51c5d76b
Aug 08 06:07:17 dist-test-slave-zj8q krb5kdc[23362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169237, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:07:17.167843 23491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:17.156622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:39687 (local address 127.1.151.190:39611)
0808 06:07:17.156764 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:17.156768 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:17.157697 (+ 929us) server_negotiation.cc:407] Connection header received
0808 06:07:17.158609 (+ 912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:17.158611 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:17.158658 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:17.158729 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:17.159821 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:17.160371 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:17.161441 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:17.161667 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:17.161747 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:17.164353 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:17.164356 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:17.164359 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:17.164385 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:17.165842 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:17.166379 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:17.166382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:17.166383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:17.166464 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:17.166795 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:17.166798 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:17.166799 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:17.166978 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:17.167060 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:17.167557 (+ 497us) server_negotiation.cc:299] Negotiation successful
0808 06:07:17.167685 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 06:07:17.168620 24164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:17.156948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.131:39687)
0808 06:07:17.157576 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:17.157614 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:17.158401 (+ 787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:17.158887 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:17.158894 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:17.159224 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:07:17.159665 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:17.159677 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:17.160462 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:17.160464 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:17.161327 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:17.161336 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:17.161814 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:17.161817 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:17.161959 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:17.162495 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:07:17.162516 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:07:17.164200 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:07:17.165960 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:17.165966 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:17.165977 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:17.166283 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:17.166548 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:17.166551 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:17.166553 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:17.166694 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:17.167081 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:07:17.167088 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:07:17.167309 (+ 221us) client_negotiation.cc:769] Sending connection context
0808 06:07:17.167464 (+ 155us) client_negotiation.cc:240] Negotiation successful
0808 06:07:17.167801 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":110,"threads_started":1}
I20260808 06:07:17.169657 24162 heartbeater.cc:347] Connected to a master server at 127.1.151.190:39611
I20260808 06:07:17.169883 24162 heartbeater.cc:464] Registering TS with master...
I20260808 06:07:17.170389 24162 heartbeater.cc:511] Master 127.1.151.190:39611 requested a full tablet report, sending...
I20260808 06:07:17.171430 23433 ts_manager.cc:194] Registered new tserver with Master: 2bbf31860ef14a1c8b7fed6f6a415153 (127.1.151.131:43389)
I20260808 06:07:17.171984 23433 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.131@KRBTEST.COM'} at 127.1.151.131:39687
I20260808 06:07:17.176928 1630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 06:07:17.185877 23491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:17.178864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36384 (local address 127.1.151.190:39611)
0808 06:07:17.178999 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:17.179003 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:17.179154 (+ 151us) server_negotiation.cc:407] Connection header received
0808 06:07:17.179297 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:17.179299 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:17.179342 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:17.179409 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:17.180064 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:17.180564 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:17.181328 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:17.181543 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:17.181649 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:17.182926 (+ 1277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:17.182930 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:17.182932 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:17.182953 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:17.184339 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:17.184740 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:17.184743 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:17.184744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:17.184825 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:17.185057 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:17.185060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:17.185061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:17.185275 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:17.185377 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:17.185615 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 06:07:17.185718 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 06:07:17.188680 23433 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36384:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 06:07:17.191668 23433 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 06:07:17.206127 24180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:17.200046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:45793 (local address 127.0.0.1:46360)
0808 06:07:17.200988 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:17.201040 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:17.201190 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:17.201699 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:17.201705 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:17.201727 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:07:17.201872 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:17.201879 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:17.203084 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:17.203087 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:17.204413 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:17.204424 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:17.205550 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:17.205553 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:17.205667 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:17.205713 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 06:07:17.205815 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 06:07:17.205973 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":770,"mutex_wait_us":25,"thread_start_us":97,"threads_started":1}
I20260808 06:07:17.206338 24182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:17.200377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57758 (local address 127.1.151.129:39351)
0808 06:07:17.200723 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:17.200739 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:17.201128 (+ 389us) server_negotiation.cc:407] Connection header received
0808 06:07:17.201383 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:17.201389 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:17.201582 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:17.201714 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:07:17.202171 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:17.203294 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:17.204638 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:17.205408 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:17.205492 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:17.205659 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:17.206073 (+ 414us) server_negotiation.cc:299] Negotiation successful
0808 06:07:17.206197 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":74,"threads_started":1}
I20260808 06:07:17.206357 24183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:17.200884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:43389 (local address 127.0.0.1:46744)
0808 06:07:17.201400 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:17.201411 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:17.201478 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:17.202495 (+ 1017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:17.202498 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:17.202516 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:07:17.202618 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:17.202625 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:17.203790 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:17.203794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:17.204946 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:17.204956 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:17.205796 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:17.205799 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:17.205901 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:17.205945 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 06:07:17.206021 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 06:07:17.206115 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"mutex_wait_us":37,"thread_start_us":87,"threads_started":1}
I20260808 06:07:17.206797 24185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:17.201125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46744 (local address 127.1.151.131:43389)
0808 06:07:17.202048 (+ 923us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:17.202060 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:17.202081 (+ 21us) server_negotiation.cc:407] Connection header received
0808 06:07:17.202170 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:17.202176 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:17.202353 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:17.202506 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:07:17.202729 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:17.203667 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:17.205062 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:17.205682 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:17.205783 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:17.205919 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:17.206547 (+ 628us) server_negotiation.cc:299] Negotiation successful
0808 06:07:17.206660 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":817,"thread_start_us":101,"threads_started":1}
I20260808 06:07:17.206357 24178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:17.200217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:39351 (local address 127.0.0.1:57758)
0808 06:07:17.200988 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:17.201040 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:17.201210 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:17.201905 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:17.201908 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:17.201925 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:07:17.202001 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:17.202006 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:17.203431 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:17.203434 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:17.204514 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:17.204523 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:17.205760 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:17.205764 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:17.205861 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:17.205946 (+ 85us) client_negotiation.cc:769] Sending connection context
0808 06:07:17.206021 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 06:07:17.206120 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"mutex_wait_us":29,"spinlock_wait_cycles":14208,"thread_start_us":104,"threads_started":1}
I20260808 06:07:17.207235 24181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:17.200180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46360 (local address 127.1.151.130:45793)
0808 06:07:17.201201 (+ 1021us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:17.201213 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:17.201237 (+ 24us) server_negotiation.cc:407] Connection header received
0808 06:07:17.201318 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:17.201323 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:17.201537 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:17.201707 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:07:17.202001 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:17.202974 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:17.204554 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:17.205408 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:17.205511 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:17.205659 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:17.206852 (+ 1193us) server_negotiation.cc:299] Negotiation successful
0808 06:07:17.207075 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":918,"thread_start_us":141,"threads_started":1}
I20260808 06:07:17.209683 24062 tablet_service.cc:1467] Processing CreateTablet for tablet 647d57c253a341919a235e86618d3a71 (DEFAULT_TABLE table=test-table [id=b06dbbc88bd7483880f90cff82f6890f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:07:17.209959 23849 tablet_service.cc:1467] Processing CreateTablet for tablet 647d57c253a341919a235e86618d3a71 (DEFAULT_TABLE table=test-table [id=b06dbbc88bd7483880f90cff82f6890f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:07:17.209964 23631 tablet_service.cc:1467] Processing CreateTablet for tablet 647d57c253a341919a235e86618d3a71 (DEFAULT_TABLE table=test-table [id=b06dbbc88bd7483880f90cff82f6890f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:07:17.210748 24062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 647d57c253a341919a235e86618d3a71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:07:17.211016 23849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 647d57c253a341919a235e86618d3a71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:07:17.211016 23631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 647d57c253a341919a235e86618d3a71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:07:17.217272 24187 tablet_bootstrap.cc:492] T 647d57c253a341919a235e86618d3a71 P 2bbf31860ef14a1c8b7fed6f6a415153: Bootstrap starting.
I20260808 06:07:17.218827 24189 tablet_bootstrap.cc:492] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2: Bootstrap starting.
I20260808 06:07:17.219447 24187 tablet_bootstrap.cc:654] T 647d57c253a341919a235e86618d3a71 P 2bbf31860ef14a1c8b7fed6f6a415153: Neither blocks nor log segments found. Creating new log.
I20260808 06:07:17.220500 24189 tablet_bootstrap.cc:654] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2: Neither blocks nor log segments found. Creating new log.
I20260808 06:07:17.220482 24187 log.cc:824] T 647d57c253a341919a235e86618d3a71 P 2bbf31860ef14a1c8b7fed6f6a415153: Log is configured to *not* fsync() on all Append() calls
I20260808 06:07:17.221460 24189 log.cc:824] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2: Log is configured to *not* fsync() on all Append() calls
I20260808 06:07:17.222128 24188 tablet_bootstrap.cc:492] T 647d57c253a341919a235e86618d3a71 P 38fc7605759a46d6b9e67bfb0d13372c: Bootstrap starting.
I20260808 06:07:17.222802 24187 tablet_bootstrap.cc:492] T 647d57c253a341919a235e86618d3a71 P 2bbf31860ef14a1c8b7fed6f6a415153: No bootstrap required, opened a new log
I20260808 06:07:17.222988 24187 ts_tablet_manager.cc:1397] T 647d57c253a341919a235e86618d3a71 P 2bbf31860ef14a1c8b7fed6f6a415153: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260808 06:07:17.223407 24189 tablet_bootstrap.cc:492] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2: No bootstrap required, opened a new log
I20260808 06:07:17.223589 24189 ts_tablet_manager.cc:1397] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 06:07:17.224684 24188 tablet_bootstrap.cc:654] T 647d57c253a341919a235e86618d3a71 P 38fc7605759a46d6b9e67bfb0d13372c: Neither blocks nor log segments found. Creating new log.
I20260808 06:07:17.225497 24188 log.cc:824] T 647d57c253a341919a235e86618d3a71 P 38fc7605759a46d6b9e67bfb0d13372c: Log is configured to *not* fsync() on all Append() calls
I20260808 06:07:17.225903 24187 raft_consensus.cc:348] T 647d57c253a341919a235e86618d3a71 P 2bbf31860ef14a1c8b7fed6f6a415153 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38fc7605759a46d6b9e67bfb0d13372c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45793 } } peers { permanent_uuid: "7ce698571e9f4da3a09c77e6ad265cd2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39351 } } peers { permanent_uuid: "2bbf31860ef14a1c8b7fed6f6a415153" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43389 } }
I20260808 06:07:17.226234 24187 raft_consensus.cc:374] T 647d57c253a341919a235e86618d3a71 P 2bbf31860ef14a1c8b7fed6f6a415153 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:07:17.226351 24187 raft_consensus.cc:729] T 647d57c253a341919a235e86618d3a71 P 2bbf31860ef14a1c8b7fed6f6a415153 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bbf31860ef14a1c8b7fed6f6a415153, State: Initialized, Role: FOLLOWER
I20260808 06:07:17.226778 24187 consensus_queue.cc:260] T 647d57c253a341919a235e86618d3a71 P 2bbf31860ef14a1c8b7fed6f6a415153 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38fc7605759a46d6b9e67bfb0d13372c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45793 } } peers { permanent_uuid: "7ce698571e9f4da3a09c77e6ad265cd2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39351 } } peers { permanent_uuid: "2bbf31860ef14a1c8b7fed6f6a415153" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43389 } }
I20260808 06:07:17.227393 24162 heartbeater.cc:503] Master 127.1.151.190:39611 was elected leader, sending a full tablet report...
I20260808 06:07:17.227316 24189 raft_consensus.cc:348] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38fc7605759a46d6b9e67bfb0d13372c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45793 } } peers { permanent_uuid: "7ce698571e9f4da3a09c77e6ad265cd2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39351 } } peers { permanent_uuid: "2bbf31860ef14a1c8b7fed6f6a415153" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43389 } }
I20260808 06:07:17.227557 24189 raft_consensus.cc:374] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:07:17.227627 24189 raft_consensus.cc:729] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ce698571e9f4da3a09c77e6ad265cd2, State: Initialized, Role: FOLLOWER
I20260808 06:07:17.227636 24188 tablet_bootstrap.cc:492] T 647d57c253a341919a235e86618d3a71 P 38fc7605759a46d6b9e67bfb0d13372c: No bootstrap required, opened a new log
I20260808 06:07:17.227807 24188 ts_tablet_manager.cc:1397] T 647d57c253a341919a235e86618d3a71 P 38fc7605759a46d6b9e67bfb0d13372c: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260808 06:07:17.228106 24189 consensus_queue.cc:260] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38fc7605759a46d6b9e67bfb0d13372c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45793 } } peers { permanent_uuid: "7ce698571e9f4da3a09c77e6ad265cd2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39351 } } peers { permanent_uuid: "2bbf31860ef14a1c8b7fed6f6a415153" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43389 } }
I20260808 06:07:17.230873 23734 heartbeater.cc:503] Master 127.1.151.190:39611 was elected leader, sending a full tablet report...
I20260808 06:07:17.231160 24189 ts_tablet_manager.cc:1428] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2: Time spent starting tablet: real 0.008s user 0.001s sys 0.007s
I20260808 06:07:17.231160 24188 raft_consensus.cc:348] T 647d57c253a341919a235e86618d3a71 P 38fc7605759a46d6b9e67bfb0d13372c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38fc7605759a46d6b9e67bfb0d13372c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45793 } } peers { permanent_uuid: "7ce698571e9f4da3a09c77e6ad265cd2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39351 } } peers { permanent_uuid: "2bbf31860ef14a1c8b7fed6f6a415153" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43389 } }
I20260808 06:07:17.231417 24188 raft_consensus.cc:374] T 647d57c253a341919a235e86618d3a71 P 38fc7605759a46d6b9e67bfb0d13372c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:07:17.231498 24188 raft_consensus.cc:729] T 647d57c253a341919a235e86618d3a71 P 38fc7605759a46d6b9e67bfb0d13372c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38fc7605759a46d6b9e67bfb0d13372c, State: Initialized, Role: FOLLOWER
I20260808 06:07:17.231943 24188 consensus_queue.cc:260] T 647d57c253a341919a235e86618d3a71 P 38fc7605759a46d6b9e67bfb0d13372c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38fc7605759a46d6b9e67bfb0d13372c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45793 } } peers { permanent_uuid: "7ce698571e9f4da3a09c77e6ad265cd2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39351 } } peers { permanent_uuid: "2bbf31860ef14a1c8b7fed6f6a415153" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43389 } }
I20260808 06:07:17.232514 24187 ts_tablet_manager.cc:1428] T 647d57c253a341919a235e86618d3a71 P 2bbf31860ef14a1c8b7fed6f6a415153: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20260808 06:07:17.233718 23955 heartbeater.cc:503] Master 127.1.151.190:39611 was elected leader, sending a full tablet report...
I20260808 06:07:17.233994 24188 ts_tablet_manager.cc:1428] T 647d57c253a341919a235e86618d3a71 P 38fc7605759a46d6b9e67bfb0d13372c: Time spent starting tablet: real 0.006s user 0.000s sys 0.005s
W20260808 06:07:17.405768 24163 tablet.cc:2369] T 647d57c253a341919a235e86618d3a71 P 2bbf31860ef14a1c8b7fed6f6a415153: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 06:07:17.442356 23956 tablet.cc:2369] T 647d57c253a341919a235e86618d3a71 P 38fc7605759a46d6b9e67bfb0d13372c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 06:07:17.457211 24197 raft_consensus.cc:482] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:07:17.457401 24197 raft_consensus.cc:504] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38fc7605759a46d6b9e67bfb0d13372c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45793 } } peers { permanent_uuid: "7ce698571e9f4da3a09c77e6ad265cd2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39351 } } peers { permanent_uuid: "2bbf31860ef14a1c8b7fed6f6a415153" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43389 } }
I20260808 06:07:17.458679 24197 leader_election.cc:288] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38fc7605759a46d6b9e67bfb0d13372c (127.1.151.130:45793), 2bbf31860ef14a1c8b7fed6f6a415153 (127.1.151.131:43389)
I20260808 06:07:17.462658 24181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:17.459198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:48119 (local address 127.1.151.130:45793)
0808 06:07:17.459328 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:17.459333 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:17.459456 (+ 123us) server_negotiation.cc:407] Connection header received
0808 06:07:17.459598 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:17.459601 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:17.459656 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:17.459728 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:07:17.460018 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:17.460557 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:17.461664 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:17.462052 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:17.462186 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:17.462243 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:17.462462 (+ 219us) server_negotiation.cc:299] Negotiation successful
0808 06:07:17.462531 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 06:07:17.462671 24185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:17.459233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:45913 (local address 127.1.151.131:43389)
0808 06:07:17.459340 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:17.459344 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:17.459627 (+ 283us) server_negotiation.cc:407] Connection header received
0808 06:07:17.459714 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:17.459717 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:17.459769 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:17.459844 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:07:17.460089 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:17.460583 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:17.461683 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:17.462072 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:17.462198 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:17.462252 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:17.462484 (+ 232us) server_negotiation.cc:299] Negotiation successful
0808 06:07:17.462550 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 06:07:17.462688 24203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:17.458888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:45793 (local address 127.1.151.129:48119)
0808 06:07:17.459348 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:17.459364 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:17.459489 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:17.459748 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:17.459750 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:17.459767 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:07:17.459907 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:17.459915 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:17.460672 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:17.460675 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:17.461542 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:17.461552 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:17.462194 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:17.462198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:17.462329 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:17.462362 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 06:07:17.462428 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 06:07:17.462501 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"mutex_wait_us":23,"thread_start_us":98,"threads_started":1}
I20260808 06:07:17.462693 24201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:17.458953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:43389 (local address 127.1.151.129:45913)
0808 06:07:17.459507 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:17.459519 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:17.459621 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:17.459862 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:17.459864 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:17.459878 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:07:17.459970 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:17.459976 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:17.460702 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:17.460705 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:17.461552 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:17.461562 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:17.462200 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:17.462204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:17.462364 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:17.462383 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 06:07:17.462454 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 06:07:17.462530 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"mutex_wait_us":103,"thread_start_us":136,"threads_started":1}
I20260808 06:07:17.463490 24094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647d57c253a341919a235e86618d3a71" candidate_uuid: "7ce698571e9f4da3a09c77e6ad265cd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bbf31860ef14a1c8b7fed6f6a415153" is_pre_election: true
I20260808 06:07:17.463521 23877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647d57c253a341919a235e86618d3a71" candidate_uuid: "7ce698571e9f4da3a09c77e6ad265cd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38fc7605759a46d6b9e67bfb0d13372c" is_pre_election: true
I20260808 06:07:17.463763 24094 raft_consensus.cc:2454] T 647d57c253a341919a235e86618d3a71 P 2bbf31860ef14a1c8b7fed6f6a415153 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ce698571e9f4da3a09c77e6ad265cd2 in term 0.
I20260808 06:07:17.463774 23877 raft_consensus.cc:2454] T 647d57c253a341919a235e86618d3a71 P 38fc7605759a46d6b9e67bfb0d13372c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ce698571e9f4da3a09c77e6ad265cd2 in term 0.
I20260808 06:07:17.464210 23552 leader_election.cc:302] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bbf31860ef14a1c8b7fed6f6a415153, 7ce698571e9f4da3a09c77e6ad265cd2; no voters:
I20260808 06:07:17.464476 24197 raft_consensus.cc:2781] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 06:07:17.464564 24197 raft_consensus.cc:482] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 06:07:17.464624 24197 raft_consensus.cc:3037] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:07:17.465639 24197 raft_consensus.cc:504] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38fc7605759a46d6b9e67bfb0d13372c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45793 } } peers { permanent_uuid: "7ce698571e9f4da3a09c77e6ad265cd2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39351 } } peers { permanent_uuid: "2bbf31860ef14a1c8b7fed6f6a415153" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43389 } }
I20260808 06:07:17.465931 24197 leader_election.cc:288] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2 [CANDIDATE]: Term 1 election: Requested vote from peers 38fc7605759a46d6b9e67bfb0d13372c (127.1.151.130:45793), 2bbf31860ef14a1c8b7fed6f6a415153 (127.1.151.131:43389)
I20260808 06:07:17.466284 24094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647d57c253a341919a235e86618d3a71" candidate_uuid: "7ce698571e9f4da3a09c77e6ad265cd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bbf31860ef14a1c8b7fed6f6a415153"
I20260808 06:07:17.466290 23877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647d57c253a341919a235e86618d3a71" candidate_uuid: "7ce698571e9f4da3a09c77e6ad265cd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38fc7605759a46d6b9e67bfb0d13372c"
I20260808 06:07:17.466401 24094 raft_consensus.cc:3037] T 647d57c253a341919a235e86618d3a71 P 2bbf31860ef14a1c8b7fed6f6a415153 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:07:17.466405 23877 raft_consensus.cc:3037] T 647d57c253a341919a235e86618d3a71 P 38fc7605759a46d6b9e67bfb0d13372c [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:07:17.467367 23877 raft_consensus.cc:2454] T 647d57c253a341919a235e86618d3a71 P 38fc7605759a46d6b9e67bfb0d13372c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ce698571e9f4da3a09c77e6ad265cd2 in term 1.
I20260808 06:07:17.467442 24094 raft_consensus.cc:2454] T 647d57c253a341919a235e86618d3a71 P 2bbf31860ef14a1c8b7fed6f6a415153 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ce698571e9f4da3a09c77e6ad265cd2 in term 1.
I20260808 06:07:17.467648 23553 leader_election.cc:302] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38fc7605759a46d6b9e67bfb0d13372c, 7ce698571e9f4da3a09c77e6ad265cd2; no voters:
I20260808 06:07:17.467813 24197 raft_consensus.cc:2781] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:07:17.468015 24197 raft_consensus.cc:686] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 1 LEADER]: Becoming Leader. State: Replica: 7ce698571e9f4da3a09c77e6ad265cd2, State: Running, Role: LEADER
I20260808 06:07:17.468309 24197 consensus_queue.cc:237] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38fc7605759a46d6b9e67bfb0d13372c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45793 } } peers { permanent_uuid: "7ce698571e9f4da3a09c77e6ad265cd2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39351 } } peers { permanent_uuid: "2bbf31860ef14a1c8b7fed6f6a415153" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43389 } }
I20260808 06:07:17.471436 23433 catalog_manager.cc:5495] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ce698571e9f4da3a09c77e6ad265cd2 (127.1.151.129). New cstate: current_term: 1 leader_uuid: "7ce698571e9f4da3a09c77e6ad265cd2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38fc7605759a46d6b9e67bfb0d13372c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ce698571e9f4da3a09c77e6ad265cd2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bbf31860ef14a1c8b7fed6f6a415153" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43389 } health_report { overall_health: UNKNOWN } } }
W20260808 06:07:17.477767 23736 tablet.cc:2369] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 06:07:17.498972 24182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:17.495487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57770 (local address 127.1.151.129:39351)
0808 06:07:17.495620 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:17.495625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:17.495640 (+ 15us) server_negotiation.cc:407] Connection header received
0808 06:07:17.495787 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:17.495789 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:17.495848 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:17.495937 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:07:17.496205 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:17.496701 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:17.497530 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:17.497752 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:17.497840 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:17.498146 (+ 306us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:07:17.498589 (+ 443us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:07:17.498698 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:17.498776 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 06:07:17.498837 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 06:07:17.504730 23877 raft_consensus.cc:1261] T 647d57c253a341919a235e86618d3a71 P 38fc7605759a46d6b9e67bfb0d13372c [term 1 FOLLOWER]: Refusing update from remote peer 7ce698571e9f4da3a09c77e6ad265cd2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:07:17.504730 24094 raft_consensus.cc:1261] T 647d57c253a341919a235e86618d3a71 P 2bbf31860ef14a1c8b7fed6f6a415153 [term 1 FOLLOWER]: Refusing update from remote peer 7ce698571e9f4da3a09c77e6ad265cd2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:07:17.505393 24204 consensus_queue.cc:1035] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38fc7605759a46d6b9e67bfb0d13372c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:07:17.505582 24197 consensus_queue.cc:1035] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bbf31860ef14a1c8b7fed6f6a415153" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:07:17.512395 24208 mvcc.cc:205] Tried to move back new op lower bound from 7316149196812722176 to 7316149196672671744. Current Snapshot: MvccSnapshot[applied={T|T < 7316149196812722176}]
I20260808 06:07:17.515184 24211 mvcc.cc:205] Tried to move back new op lower bound from 7316149196812722176 to 7316149196672671744. Current Snapshot: MvccSnapshot[applied={T|T < 7316149196812722176}]
I20260808 06:07:17.520789 23433 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:07:17.520933 23433 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:07:17.523254 23433 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6: Sending DeleteTablet for 3 replicas of tablet 647d57c253a341919a235e86618d3a71
I20260808 06:07:17.524065 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 23525
I20260808 06:07:17.524044 23631 tablet_service.cc:1514] Processing DeleteTablet for tablet 647d57c253a341919a235e86618d3a71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:07:17 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:57758
I20260808 06:07:17.524056 23849 tablet_service.cc:1514] Processing DeleteTablet for tablet 647d57c253a341919a235e86618d3a71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:07:17 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:46360
I20260808 06:07:17.534564 24062 tablet_service.cc:1514] Processing DeleteTablet for tablet 647d57c253a341919a235e86618d3a71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:07:17 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:46744
I20260808 06:07:17.534708 24223 tablet_replica.cc:330] T 647d57c253a341919a235e86618d3a71 P 38fc7605759a46d6b9e67bfb0d13372c: stopping tablet replica
I20260808 06:07:17.535029 24225 tablet_replica.cc:330] T 647d57c253a341919a235e86618d3a71 P 2bbf31860ef14a1c8b7fed6f6a415153: stopping tablet replica
I20260808 06:07:17.535061 24223 raft_consensus.cc:2229] T 647d57c253a341919a235e86618d3a71 P 38fc7605759a46d6b9e67bfb0d13372c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:07:17.535292 24225 raft_consensus.cc:2229] T 647d57c253a341919a235e86618d3a71 P 2bbf31860ef14a1c8b7fed6f6a415153 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:07:17.535408 24223 raft_consensus.cc:2258] T 647d57c253a341919a235e86618d3a71 P 38fc7605759a46d6b9e67bfb0d13372c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:07:17.535573 24225 raft_consensus.cc:2258] T 647d57c253a341919a235e86618d3a71 P 2bbf31860ef14a1c8b7fed6f6a415153 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:07:17.537032 24225 ts_tablet_manager.cc:1905] T 647d57c253a341919a235e86618d3a71 P 2bbf31860ef14a1c8b7fed6f6a415153: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:07:17.537086 24223 ts_tablet_manager.cc:1905] T 647d57c253a341919a235e86618d3a71 P 38fc7605759a46d6b9e67bfb0d13372c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:07:17.539258 24225 ts_tablet_manager.cc:1918] T 647d57c253a341919a235e86618d3a71 P 2bbf31860ef14a1c8b7fed6f6a415153: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:07:17.539350 24225 log.cc:1196] T 647d57c253a341919a235e86618d3a71 P 2bbf31860ef14a1c8b7fed6f6a415153: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/wal/wals/647d57c253a341919a235e86618d3a71
I20260808 06:07:17.539642 24225 ts_tablet_manager.cc:1939] T 647d57c253a341919a235e86618d3a71 P 2bbf31860ef14a1c8b7fed6f6a415153: Deleting consensus metadata
I20260808 06:07:17.539769 24223 ts_tablet_manager.cc:1918] T 647d57c253a341919a235e86618d3a71 P 38fc7605759a46d6b9e67bfb0d13372c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:07:17.539867 24223 log.cc:1196] T 647d57c253a341919a235e86618d3a71 P 38fc7605759a46d6b9e67bfb0d13372c: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/wal/wals/647d57c253a341919a235e86618d3a71
I20260808 06:07:17.540133 24223 ts_tablet_manager.cc:1939] T 647d57c253a341919a235e86618d3a71 P 38fc7605759a46d6b9e67bfb0d13372c: Deleting consensus metadata
I20260808 06:07:17.540773 23415 catalog_manager.cc:4841] TS 2bbf31860ef14a1c8b7fed6f6a415153 (127.1.151.131:43389): tablet 647d57c253a341919a235e86618d3a71 (table test-table [id=b06dbbc88bd7483880f90cff82f6890f]) successfully deleted
I20260808 06:07:17.541280 23416 catalog_manager.cc:4841] TS 38fc7605759a46d6b9e67bfb0d13372c (127.1.151.130:45793): tablet 647d57c253a341919a235e86618d3a71 (table test-table [id=b06dbbc88bd7483880f90cff82f6890f]) successfully deleted
W20260808 06:07:17.541741 23415 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.151.129:39351 (error 108)
I20260808 06:07:17.541942 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 23742
I20260808 06:07:17.545253 24183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:17.542432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:39351 (local address 127.0.0.1:57772)
0808 06:07:17.545051 (+ 2619us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:17.545159 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.129:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2501}
W20260808 06:07:17.545401 23415 catalog_manager.cc:4584] TS 7ce698571e9f4da3a09c77e6ad265cd2 (127.1.151.129:39351): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 647d57c253a341919a235e86618d3a71: Network error: Client connection negotiation failed: client connection to 127.1.151.129:39351: connect: Connection refused (error 111)
I20260808 06:07:17.553472 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 23965
I20260808 06:07:17.565694 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 23383
I20260808 06:07:17.578495 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:39611
--webserver_interface=127.1.151.190
--webserver_port=41435
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:34817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:39611
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:07:17.688328 24227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:07:17.688657 24227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:07:17.688727 24227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:07:17.692946 24227 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:07:17.692996 24227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:07:17.693014 24227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:07:17.693030 24227 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:07:17.693058 24227 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:07:17.698108 24227 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:34817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:39611
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:39611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=41435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:07:17.699152 24227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:07:17.700202 24227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:07:17.713747 24234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:17.713752 24235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:17.713753 24240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:17.713918 24227 server_base.cc:1034] running on GCE node
I20260808 06:07:17.714696 24227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:07:17.715764 24227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:07:17.716913 24227 hybrid_clock.cc:648] HybridClock initialized: now 1786169237716885 us; error 38 us; skew 500 ppm
Aug 08 06:07:17 dist-test-slave-zj8q krb5kdc[23362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169237, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:07:17.719583 24227 init.cc:377] Logged in from keytab as kudu/127.1.151.190@KRBTEST.COM (short username kudu)
I20260808 06:07:17.720425 24227 webserver.cc:460] Webserver started at http://127.1.151.190:41435/ using document root <none> and password file <none>
I20260808 06:07:17.720966 24227 fs_manager.cc:359] Metadata directory not provided
I20260808 06:07:17.721066 24227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:07:17.725811 24227 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 06:07:17.727883 24254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:17.729038 24227 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260808 06:07:17.729162 24227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "51bb59562e294210867a4e6920f254b6"
format_stamp: "Formatted at 2026-08-08 06:07:16 on dist-test-slave-zj8q"
server_key: "040f1b8315b7842701fb291a6e279abb"
server_key_iv: "5a2b5e974e7e7606478bb0600ab4c863"
server_key_version: "encryptionkey@0"
I20260808 06:07:17.729729 24227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:07:17.750923 24227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:07:17.753562 24227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:07:17.753736 24227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:07:17.760490 24227 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:39611
I20260808 06:07:17.760761 24331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:39611 every 8 connection(s)
I20260808 06:07:17.761543 24227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:07:17.765251 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 24227
I20260808 06:07:17.765812 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.129:39351
--local_ip_for_outbound_sockets=127.1.151.129
--tserver_master_addrs=127.1.151.190:39611
--webserver_port=32961
--webserver_interface=127.1.151.129
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:34817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 06:07:17.766247 24333 sys_catalog.cc:263] Verifying existing consensus state
I20260808 06:07:17.768265 24333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6: Bootstrap starting.
I20260808 06:07:17.786221 24333 log.cc:824] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6: Log is configured to *not* fsync() on all Append() calls
I20260808 06:07:17.794950 24333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 06:07:17.795282 24333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6: Bootstrap complete.
I20260808 06:07:17.799209 24333 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51bb59562e294210867a4e6920f254b6" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39611 } }
I20260808 06:07:17.799583 24333 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51bb59562e294210867a4e6920f254b6, State: Initialized, Role: FOLLOWER
I20260808 06:07:17.800068 24333 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51bb59562e294210867a4e6920f254b6" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39611 } }
I20260808 06:07:17.800194 24333 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:07:17.800256 24333 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:07:17.800349 24333 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [term 1 FOLLOWER]: Advancing to term 2
I20260808 06:07:17.802271 24333 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51bb59562e294210867a4e6920f254b6" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39611 } }
I20260808 06:07:17.802582 24333 leader_election.cc:302] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51bb59562e294210867a4e6920f254b6; no voters:
I20260808 06:07:17.802882 24333 leader_election.cc:288] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 06:07:17.802978 24342 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 06:07:17.803182 24342 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [term 2 LEADER]: Becoming Leader. State: Replica: 51bb59562e294210867a4e6920f254b6, State: Running, Role: LEADER
I20260808 06:07:17.803467 24342 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51bb59562e294210867a4e6920f254b6" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39611 } }
I20260808 06:07:17.804092 24333 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:07:17.804426 24344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51bb59562e294210867a4e6920f254b6. Latest consensus state: current_term: 2 leader_uuid: "51bb59562e294210867a4e6920f254b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51bb59562e294210867a4e6920f254b6" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39611 } } }
I20260808 06:07:17.804553 24344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [sys.catalog]: This master's current role is: LEADER
I20260808 06:07:17.804798 24343 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "51bb59562e294210867a4e6920f254b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51bb59562e294210867a4e6920f254b6" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39611 } } }
I20260808 06:07:17.804899 24343 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [sys.catalog]: This master's current role is: LEADER
I20260808 06:07:17.804934 24353 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:07:17.809662 24353 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:07:17.810370 24353 catalog_manager.cc:1259] Loaded cluster ID: 7bdd4ae20a4e4ac18482b0cdadc08061
I20260808 06:07:17.810431 24353 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:07:17.811885 24353 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 06:07:17.816627 24353 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 06:07:17.821818 24353 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:07:17.822571 24353 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6: Loaded TSK: 0
I20260808 06:07:17.823752 24353 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 06:07:17.880493 24339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:07:17.880730 24339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:07:17.880790 24339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:07:17.884728 24339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:07:17.884783 24339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:07:17.884856 24339 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.129
I20260808 06:07:17.889870 24339 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:34817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.129:39351
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.151.129
--webserver_port=32961
--tserver_master_addrs=127.1.151.190:39611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.129
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:07:17.891014 24339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:07:17.892138 24339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:07:17.902697 24383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:17.902791 24339 server_base.cc:1034] running on GCE node
W20260808 06:07:17.902722 24382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:17.902697 24385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:17.903362 24339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:07:17.903980 24339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:07:17.905198 24339 hybrid_clock.cc:648] HybridClock initialized: now 1786169237905163 us; error 51 us; skew 500 ppm
Aug 08 06:07:17 dist-test-slave-zj8q krb5kdc[23362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169237, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:07:17.908138 24339 init.cc:377] Logged in from keytab as kudu/127.1.151.129@KRBTEST.COM (short username kudu)
I20260808 06:07:17.909015 24339 webserver.cc:460] Webserver started at http://127.1.151.129:32961/ using document root <none> and password file <none>
I20260808 06:07:17.909540 24339 fs_manager.cc:359] Metadata directory not provided
I20260808 06:07:17.909663 24339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:07:17.913532 24339 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 06:07:17.915751 24392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:17.917078 24339 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 06:07:17.917191 24339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/wal
uuid: "7ce698571e9f4da3a09c77e6ad265cd2"
format_stamp: "Formatted at 2026-08-08 06:07:16 on dist-test-slave-zj8q"
server_key: "dbff06773774fe574da4b5b669a147ba"
server_key_iv: "ffba471894e68b82de61319530746781"
server_key_version: "encryptionkey@0"
I20260808 06:07:17.917732 24339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:07:17.938289 24339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:07:17.941257 24339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:07:17.941431 24339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:07:17.942013 24339 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:07:17.943272 24401 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 06:07:17.947587 24339 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 06:07:17.947662 24339 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260808 06:07:17.947741 24339 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 06:07:17.949440 24339 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 06:07:17.949487 24339 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260808 06:07:17.949774 24401 tablet_bootstrap.cc:492] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2: Bootstrap starting.
I20260808 06:07:17.959194 24339 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.129:39351
I20260808 06:07:17.959270 24570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.129:39351 every 8 connection(s)
I20260808 06:07:17.960130 24339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
I20260808 06:07:17.963549 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 24339
I20260808 06:07:17.963948 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.130:45793
--local_ip_for_outbound_sockets=127.1.151.130
--tserver_master_addrs=127.1.151.190:39611
--webserver_port=33379
--webserver_interface=127.1.151.130
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:34817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 06:07:17.969188 24401 log.cc:824] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2: Log is configured to *not* fsync() on all Append() calls
Aug 08 06:07:17 dist-test-slave-zj8q krb5kdc[23362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169237, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:07:17.977425 24401 tablet_bootstrap.cc:492] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 06:07:17.977820 24401 tablet_bootstrap.cc:492] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2: Bootstrap complete.
I20260808 06:07:17.979265 24401 ts_tablet_manager.cc:1397] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2: Time spent bootstrapping tablet: real 0.030s user 0.022s sys 0.006s
I20260808 06:07:17.979462 24576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:17.962323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.129:59811)
0808 06:07:17.963024 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:17.963083 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:17.964122 (+ 1039us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:17.965564 (+ 1442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:17.965573 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:17.966098 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:07:17.966817 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:17.966843 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:17.968268 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:17.968271 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:17.969572 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:17.969585 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:17.970406 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:17.970410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:17.970609 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:17.971263 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:07:17.971284 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:07:17.973247 (+ 1963us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:07:17.975727 (+ 2480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:17.975733 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:17.975747 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:17.976108 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:17.976498 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:17.976501 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:17.976503 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:17.976744 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:17.977594 (+ 850us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:07:17.977600 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:07:17.977905 (+ 305us) client_negotiation.cc:769] Sending connection context
0808 06:07:17.978128 (+ 223us) client_negotiation.cc:240] Negotiation successful
0808 06:07:17.978502 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":124,"threads_started":1}
I20260808 06:07:17.979817 24574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:17.962341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:59811 (local address 127.1.151.190:39611)
0808 06:07:17.962773 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:17.962793 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:17.963201 (+ 408us) server_negotiation.cc:407] Connection header received
0808 06:07:17.964677 (+ 1476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:17.964693 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:17.965065 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:17.965403 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:17.967028 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:17.968106 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:17.969710 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:17.970264 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:17.970437 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:17.973471 (+ 3034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:17.973477 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:17.973492 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:17.973524 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:17.975591 (+ 2067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:17.976231 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:17.976237 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:17.976243 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:17.976393 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:17.976897 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:17.976901 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:17.976903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:17.977385 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:17.977525 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:17.978289 (+ 764us) server_negotiation.cc:299] Negotiation successful
0808 06:07:17.978579 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":156,"threads_started":1}
I20260808 06:07:17.981835 24401 raft_consensus.cc:348] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38fc7605759a46d6b9e67bfb0d13372c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45793 } } peers { permanent_uuid: "7ce698571e9f4da3a09c77e6ad265cd2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39351 } } peers { permanent_uuid: "2bbf31860ef14a1c8b7fed6f6a415153" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43389 } }
I20260808 06:07:17.982367 24401 raft_consensus.cc:729] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ce698571e9f4da3a09c77e6ad265cd2, State: Initialized, Role: FOLLOWER
I20260808 06:07:17.982786 24401 consensus_queue.cc:260] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38fc7605759a46d6b9e67bfb0d13372c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45793 } } peers { permanent_uuid: "7ce698571e9f4da3a09c77e6ad265cd2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39351 } } peers { permanent_uuid: "2bbf31860ef14a1c8b7fed6f6a415153" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43389 } }
I20260808 06:07:17.982980 24571 heartbeater.cc:347] Connected to a master server at 127.1.151.190:39611
I20260808 06:07:17.983214 24571 heartbeater.cc:464] Registering TS with master...
I20260808 06:07:17.983693 24401 ts_tablet_manager.cc:1428] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2: Time spent starting tablet: real 0.004s user 0.003s sys 0.002s
I20260808 06:07:17.983731 24571 heartbeater.cc:511] Master 127.1.151.190:39611 requested a full tablet report, sending...
I20260808 06:07:17.986231 24275 ts_manager.cc:194] Registered new tserver with Master: 7ce698571e9f4da3a09c77e6ad265cd2 (127.1.151.129:39351)
I20260808 06:07:17.989729 24275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.129@KRBTEST.COM'} at 127.1.151.129:59811
I20260808 06:07:17.992493 24571 heartbeater.cc:503] Master 127.1.151.190:39611 was elected leader, sending a full tablet report...
Aug 08 06:07:17 dist-test-slave-zj8q krb5kdc[23362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169237, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for kudu/127.1.151.129@KRBTEST.COM
I20260808 06:07:17.998487 24585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:17.988948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:39351 (local address 127.0.0.1:57786)
0808 06:07:17.989358 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:17.989401 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:17.989517 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:17.990137 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:17.990144 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:17.990577 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:07:17.990719 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:17.990729 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:17.991655 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:17.991658 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:17.992276 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:17.992286 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:17.992881 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:17.992885 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:17.992998 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:17.993562 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:07:17.993576 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:07:17.995207 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:07:17.996861 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:17.996870 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:17.996876 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:17.997090 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:17.997371 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:17.997374 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:17.997376 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:17.997462 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:17.997880 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:07:17.997886 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:07:17.997996 (+ 110us) client_negotiation.cc:769] Sending connection context
0808 06:07:17.998110 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 06:07:17.998316 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":125,"threads_started":1}
I20260808 06:07:17.998523 24587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:17.989020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57786 (local address 127.1.151.129:39351)
0808 06:07:17.989681 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:17.989692 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:17.989714 (+ 22us) server_negotiation.cc:407] Connection header received
0808 06:07:17.989778 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:17.989783 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:17.989970 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:17.990113 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:17.990825 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:17.991537 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:17.992392 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:17.992771 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:17.992873 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:17.995341 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:17.995350 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:17.995359 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:17.995380 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:17.996753 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:17.997201 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:17.997206 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:17.997207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:17.997283 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:17.997543 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:17.997546 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:17.997548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:17.997768 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:17.997885 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:17.998235 (+ 350us) server_negotiation.cc:299] Negotiation successful
0808 06:07:17.998389 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":85,"threads_started":1}
I20260808 06:07:17.999624 24468 tablet_service.cc:1514] Processing DeleteTablet for tablet 647d57c253a341919a235e86618d3a71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:07:17 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:57786
I20260808 06:07:18.000411 24588 tablet_replica.cc:330] stopping tablet replica
I20260808 06:07:18.000725 24588 raft_consensus.cc:2229] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:07:18.000969 24588 raft_consensus.cc:2258] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:07:18.002058 24588 ts_tablet_manager.cc:1905] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:07:18.006821 24588 ts_tablet_manager.cc:1918] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:07:18.006928 24588 log.cc:1196] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/wal/wals/647d57c253a341919a235e86618d3a71
I20260808 06:07:18.007230 24588 ts_tablet_manager.cc:1939] T 647d57c253a341919a235e86618d3a71 P 7ce698571e9f4da3a09c77e6ad265cd2: Deleting consensus metadata
I20260808 06:07:18.009126 24258 catalog_manager.cc:4841] TS 7ce698571e9f4da3a09c77e6ad265cd2 (127.1.151.129:39351): tablet 647d57c253a341919a235e86618d3a71 (table test-table [id=b06dbbc88bd7483880f90cff82f6890f]) successfully deleted
W20260808 06:07:18.087116 24577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:07:18.087343 24577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:07:18.087384 24577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:07:18.091233 24577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:07:18.091282 24577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:07:18.091336 24577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.130
I20260808 06:07:18.096333 24577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:34817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.130:45793
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.151.130
--webserver_port=33379
--tserver_master_addrs=127.1.151.190:39611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.130
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:07:18.097211 24577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:07:18.098340 24577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:07:18.108006 24600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:18.108016 24602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:18.108026 24599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:18.108377 24577 server_base.cc:1034] running on GCE node
I20260808 06:07:18.108770 24577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:07:18.109373 24577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:07:18.110536 24577 hybrid_clock.cc:648] HybridClock initialized: now 1786169238110515 us; error 27 us; skew 500 ppm
Aug 08 06:07:18 dist-test-slave-zj8q krb5kdc[23362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169238, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:07:18.113140 24577 init.cc:377] Logged in from keytab as kudu/127.1.151.130@KRBTEST.COM (short username kudu)
I20260808 06:07:18.113960 24577 webserver.cc:460] Webserver started at http://127.1.151.130:33379/ using document root <none> and password file <none>
I20260808 06:07:18.114524 24577 fs_manager.cc:359] Metadata directory not provided
I20260808 06:07:18.114600 24577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:07:18.118273 24577 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 06:07:18.120333 24613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:18.121595 24577 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:07:18.121698 24577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/wal
uuid: "38fc7605759a46d6b9e67bfb0d13372c"
format_stamp: "Formatted at 2026-08-08 06:07:16 on dist-test-slave-zj8q"
server_key: "47e94270c951564af4414842d69126ca"
server_key_iv: "d47dd0cb305710dbfdacea7d7112ae8f"
server_key_version: "encryptionkey@0"
I20260808 06:07:18.122243 24577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:07:18.133637 24577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:07:18.136454 24577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:07:18.136595 24577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:07:18.137090 24577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:07:18.137828 24577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:07:18.137882 24577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:18.137923 24577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:07:18.137943 24577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:18.146298 24577 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.130:45793
I20260808 06:07:18.146345 24801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.130:45793 every 8 connection(s)
I20260808 06:07:18.147223 24577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
I20260808 06:07:18.153842 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 24577
I20260808 06:07:18.154249 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.131:43389
--local_ip_for_outbound_sockets=127.1.151.131
--tserver_master_addrs=127.1.151.190:39611
--webserver_port=33361
--webserver_interface=127.1.151.131
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:34817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 06:07:18 dist-test-slave-zj8q krb5kdc[23362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169238, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:07:18.159667 24574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:18.148846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:60113 (local address 127.1.151.190:39611)
0808 06:07:18.149014 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:18.149019 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:18.149995 (+ 976us) server_negotiation.cc:407] Connection header received
0808 06:07:18.150833 (+ 838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:18.150836 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:18.150887 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:18.150959 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:18.152021 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:18.152560 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:18.153663 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:18.153888 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:18.153975 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:18.156358 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:18.156362 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:18.156364 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:18.156388 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:18.157706 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:18.158205 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:18.158208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:18.158209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:18.158288 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:18.158600 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:18.158603 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:18.158605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:18.158786 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:18.158875 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:18.159360 (+ 485us) server_negotiation.cc:299] Negotiation successful
0808 06:07:18.159500 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 06:07:18.160429 24807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:18.149230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.130:60113)
0808 06:07:18.149852 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:18.149890 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:18.150644 (+ 754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:18.151102 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:18.151109 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:18.151432 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:07:18.151902 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:18.151916 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:18.152680 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:18.152683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:18.153544 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:18.153553 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:18.154041 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:18.154045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:18.154250 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:18.154780 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:07:18.154797 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:07:18.156184 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:07:18.157821 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:18.157827 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:18.157841 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:18.158086 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:18.158373 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:18.158376 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:18.158378 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:18.158512 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:18.158889 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:07:18.158895 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:07:18.159132 (+ 237us) client_negotiation.cc:769] Sending connection context
0808 06:07:18.159300 (+ 168us) client_negotiation.cc:240] Negotiation successful
0808 06:07:18.159602 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":113,"threads_started":1}
I20260808 06:07:18.161587 24805 heartbeater.cc:347] Connected to a master server at 127.1.151.190:39611
I20260808 06:07:18.161856 24805 heartbeater.cc:464] Registering TS with master...
I20260808 06:07:18.162479 24805 heartbeater.cc:511] Master 127.1.151.190:39611 requested a full tablet report, sending...
I20260808 06:07:18.163674 24275 ts_manager.cc:194] Registered new tserver with Master: 38fc7605759a46d6b9e67bfb0d13372c (127.1.151.130:45793)
I20260808 06:07:18.164443 24275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.130@KRBTEST.COM'} at 127.1.151.130:60113
W20260808 06:07:18.265708 24809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:07:18.265961 24809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:07:18.266002 24809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:07:18.269891 24809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:07:18.269948 24809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:07:18.270004 24809 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.131
I20260808 06:07:18.275110 24809 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:34817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.131:43389
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.151.131
--webserver_port=33361
--tserver_master_addrs=127.1.151.190:39611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.131
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:07:18.276034 24809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:07:18.276980 24809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:07:18.286253 24818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:18.286283 24822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:18.286253 24817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:18.286548 24809 server_base.cc:1034] running on GCE node
I20260808 06:07:18.286940 24809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:07:18.287524 24809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:07:18.288710 24809 hybrid_clock.cc:648] HybridClock initialized: now 1786169238288694 us; error 29 us; skew 500 ppm
Aug 08 06:07:18 dist-test-slave-zj8q krb5kdc[23362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169238, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:07:18.291383 24809 init.cc:377] Logged in from keytab as kudu/127.1.151.131@KRBTEST.COM (short username kudu)
I20260808 06:07:18.292207 24809 webserver.cc:460] Webserver started at http://127.1.151.131:33361/ using document root <none> and password file <none>
I20260808 06:07:18.292718 24809 fs_manager.cc:359] Metadata directory not provided
I20260808 06:07:18.292796 24809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:07:18.296399 24809 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:07:18.298348 24832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:18.299482 24809 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:07:18.299588 24809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/wal
uuid: "2bbf31860ef14a1c8b7fed6f6a415153"
format_stamp: "Formatted at 2026-08-08 06:07:17 on dist-test-slave-zj8q"
server_key: "8180099d00e94bae7912f5617beffd41"
server_key_iv: "cc4a4e57cb71a2be30cfc51900df39be"
server_key_version: "encryptionkey@0"
I20260808 06:07:18.300091 24809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:07:18.319773 24809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:07:18.322528 24809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:07:18.322690 24809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:07:18.323240 24809 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:07:18.324013 24809 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:07:18.324061 24809 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:18.324120 24809 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:07:18.324146 24809 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:18.332242 24809 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.131:43389
I20260808 06:07:18.332305 25014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.131:43389 every 8 connection(s)
I20260808 06:07:18.333201 24809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
I20260808 06:07:18.341373 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 24809
Aug 08 06:07:18 dist-test-slave-zj8q krb5kdc[23362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169238, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:07:18.346835 24574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:18.334951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:46363 (local address 127.1.151.190:39611)
0808 06:07:18.335083 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:18.335087 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:18.336133 (+ 1046us) server_negotiation.cc:407] Connection header received
0808 06:07:18.336982 (+ 849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:18.336986 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:18.337044 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:18.337128 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:18.338311 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:18.338938 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:18.340084 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:18.340320 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:18.340408 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:18.343141 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:18.343145 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:18.343148 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:18.343173 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:18.344639 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:18.345186 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:18.345189 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:18.345191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:18.345271 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:18.345639 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:18.345642 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:18.345644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:18.345883 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:18.345971 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:18.346603 (+ 632us) server_negotiation.cc:299] Negotiation successful
0808 06:07:18.346711 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":42}
I20260808 06:07:18.347712 25017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:18.335333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.131:46363)
0808 06:07:18.335973 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:18.336011 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:18.336789 (+ 778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:18.337306 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:18.337313 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:18.337673 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:07:18.338143 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:18.338185 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:18.339071 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:18.339074 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:18.339944 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:18.339954 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:18.340461 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:18.340465 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:18.340626 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:18.341128 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:07:18.341149 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:07:18.342962 (+ 1813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:07:18.344738 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:18.344744 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:18.344758 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:18.345090 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:18.345353 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:18.345355 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:18.345357 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:18.345554 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:18.345976 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:07:18.345982 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:07:18.346291 (+ 309us) client_negotiation.cc:769] Sending connection context
0808 06:07:18.346492 (+ 201us) client_negotiation.cc:240] Negotiation successful
0808 06:07:18.346850 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":101,"threads_started":1}
I20260808 06:07:18.348810 25015 heartbeater.cc:347] Connected to a master server at 127.1.151.190:39611
I20260808 06:07:18.349048 25015 heartbeater.cc:464] Registering TS with master...
I20260808 06:07:18.349545 25015 heartbeater.cc:511] Master 127.1.151.190:39611 requested a full tablet report, sending...
I20260808 06:07:18.350396 25021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:18.342123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36400 (local address 127.1.151.190:39611)
0808 06:07:18.342473 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:18.342478 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:18.342492 (+ 14us) server_negotiation.cc:407] Connection header received
0808 06:07:18.342633 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:18.342636 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:18.342678 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:18.342749 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:18.343651 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:18.344417 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:18.345452 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:18.345784 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:18.345879 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:18.347302 (+ 1423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:18.347305 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:18.347307 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:18.347328 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:18.348836 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:18.349266 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:18.349270 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:18.349271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:18.349362 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:18.349591 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:18.349594 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:18.349596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:18.349798 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:18.349992 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:18.350098 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 06:07:18.350245 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":279,"thread_start_us":80,"threads_started":1}
I20260808 06:07:18.350703 24275 ts_manager.cc:194] Registered new tserver with Master: 2bbf31860ef14a1c8b7fed6f6a415153 (127.1.151.131:43389)
I20260808 06:07:18.351286 24275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.131@KRBTEST.COM'} at 127.1.151.131:46363
I20260808 06:07:18.352041 1630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 06:07:18.356012 25021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:18.352627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36416 (local address 127.1.151.190:39611)
0808 06:07:18.352739 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:18.352743 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:18.352928 (+ 185us) server_negotiation.cc:407] Connection header received
0808 06:07:18.353060 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:18.353063 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:18.353159 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:18.353241 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:07:18.353454 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:18.353923 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:18.354782 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:18.355002 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:18.355089 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:18.355368 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:07:18.355683 (+ 315us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:07:18.355784 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:18.355854 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 06:07:18.355908 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 06:07:18.356567 24274 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:36416:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 06:07:18.359690 24274 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 06:07:18.366837 24468 tablet_service.cc:1467] Processing CreateTablet for tablet 0fb41d8963c949eeb767ee5f78289c54 (DEFAULT_TABLE table=test-table [id=f7fa43f796c14c52937401ec150c5942]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:07:18.367509 24468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fb41d8963c949eeb767ee5f78289c54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:07:18.370316 24401 tablet_bootstrap.cc:492] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2: Bootstrap starting.
I20260808 06:07:18.370918 24585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:18.365657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:43389 (local address 127.0.0.1:46760)
0808 06:07:18.365820 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:18.365834 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:18.365903 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:18.367176 (+ 1273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:18.367179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:18.367198 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:07:18.367282 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:18.367287 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:18.368642 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:18.368646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:18.369632 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:18.369640 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:18.370517 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:18.370521 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:18.370615 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:18.370671 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 06:07:18.370725 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 06:07:18.370802 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":1}
I20260808 06:07:18.371073 25030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:18.365978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:45793 (local address 127.0.0.1:46374)
0808 06:07:18.366348 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:18.366360 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:18.366451 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:18.366955 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:18.366958 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:18.366975 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:07:18.367061 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:18.367067 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:18.368189 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:18.368192 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:18.369352 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:18.369364 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:18.370649 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:18.370652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:18.370723 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:18.370741 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 06:07:18.370804 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 06:07:18.370875 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":21,"thread_start_us":84,"threads_started":1}
I20260808 06:07:18.371084 25031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:18.366239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46374 (local address 127.1.151.130:45793)
0808 06:07:18.366564 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:18.366576 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:18.366596 (+ 20us) server_negotiation.cc:407] Connection header received
0808 06:07:18.366668 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:18.366672 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:18.366824 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:18.366957 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:07:18.367154 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:18.367938 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:18.369492 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:18.370339 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:18.370431 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:18.370575 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:18.370856 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 06:07:18.370952 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":100,"threads_started":1}
I20260808 06:07:18.371213 24401 tablet_bootstrap.cc:654] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2: Neither blocks nor log segments found. Creating new log.
I20260808 06:07:18.371627 25029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:18.365810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46760 (local address 127.1.151.131:43389)
0808 06:07:18.366659 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:18.366670 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:18.366691 (+ 21us) server_negotiation.cc:407] Connection header received
0808 06:07:18.366755 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:18.366760 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:18.366933 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:18.367416 (+ 483us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:07:18.367545 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:18.368499 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:18.369739 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:18.370385 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:18.371188 (+ 803us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:18.371330 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:18.371384 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 06:07:18.371478 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":746,"thread_start_us":151,"threads_started":1}
I20260808 06:07:18.372704 24401 tablet_bootstrap.cc:492] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2: No bootstrap required, opened a new log
I20260808 06:07:18.372793 24401 ts_tablet_manager.cc:1397] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 06:07:18.373234 24401 raft_consensus.cc:348] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bbf31860ef14a1c8b7fed6f6a415153" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43389 } } peers { permanent_uuid: "38fc7605759a46d6b9e67bfb0d13372c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45793 } } peers { permanent_uuid: "7ce698571e9f4da3a09c77e6ad265cd2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39351 } }
I20260808 06:07:18.373358 24401 raft_consensus.cc:374] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:07:18.373395 24401 raft_consensus.cc:729] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ce698571e9f4da3a09c77e6ad265cd2, State: Initialized, Role: FOLLOWER
I20260808 06:07:18.373553 24401 consensus_queue.cc:260] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bbf31860ef14a1c8b7fed6f6a415153" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43389 } } peers { permanent_uuid: "38fc7605759a46d6b9e67bfb0d13372c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45793 } } peers { permanent_uuid: "7ce698571e9f4da3a09c77e6ad265cd2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39351 } }
I20260808 06:07:18.373899 24401 ts_tablet_manager.cc:1428] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 06:07:18.373978 24690 tablet_service.cc:1467] Processing CreateTablet for tablet 0fb41d8963c949eeb767ee5f78289c54 (DEFAULT_TABLE table=test-table [id=f7fa43f796c14c52937401ec150c5942]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:07:18.374454 24907 tablet_service.cc:1467] Processing CreateTablet for tablet 0fb41d8963c949eeb767ee5f78289c54 (DEFAULT_TABLE table=test-table [id=f7fa43f796c14c52937401ec150c5942]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:07:18.375015 24690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fb41d8963c949eeb767ee5f78289c54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:07:18.375471 24907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fb41d8963c949eeb767ee5f78289c54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:07:18.381738 25033 tablet_bootstrap.cc:492] T 0fb41d8963c949eeb767ee5f78289c54 P 2bbf31860ef14a1c8b7fed6f6a415153: Bootstrap starting.
I20260808 06:07:18.382258 25036 tablet_bootstrap.cc:492] T 0fb41d8963c949eeb767ee5f78289c54 P 38fc7605759a46d6b9e67bfb0d13372c: Bootstrap starting.
I20260808 06:07:18.384016 25033 tablet_bootstrap.cc:654] T 0fb41d8963c949eeb767ee5f78289c54 P 2bbf31860ef14a1c8b7fed6f6a415153: Neither blocks nor log segments found. Creating new log.
I20260808 06:07:18.384245 25036 tablet_bootstrap.cc:654] T 0fb41d8963c949eeb767ee5f78289c54 P 38fc7605759a46d6b9e67bfb0d13372c: Neither blocks nor log segments found. Creating new log.
I20260808 06:07:18.384842 25033 log.cc:824] T 0fb41d8963c949eeb767ee5f78289c54 P 2bbf31860ef14a1c8b7fed6f6a415153: Log is configured to *not* fsync() on all Append() calls
I20260808 06:07:18.385020 25036 log.cc:824] T 0fb41d8963c949eeb767ee5f78289c54 P 38fc7605759a46d6b9e67bfb0d13372c: Log is configured to *not* fsync() on all Append() calls
I20260808 06:07:18.386684 25036 tablet_bootstrap.cc:492] T 0fb41d8963c949eeb767ee5f78289c54 P 38fc7605759a46d6b9e67bfb0d13372c: No bootstrap required, opened a new log
I20260808 06:07:18.386708 25033 tablet_bootstrap.cc:492] T 0fb41d8963c949eeb767ee5f78289c54 P 2bbf31860ef14a1c8b7fed6f6a415153: No bootstrap required, opened a new log
I20260808 06:07:18.386855 25036 ts_tablet_manager.cc:1397] T 0fb41d8963c949eeb767ee5f78289c54 P 38fc7605759a46d6b9e67bfb0d13372c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260808 06:07:18.386864 25033 ts_tablet_manager.cc:1397] T 0fb41d8963c949eeb767ee5f78289c54 P 2bbf31860ef14a1c8b7fed6f6a415153: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.001s
I20260808 06:07:18.389496 25036 raft_consensus.cc:348] T 0fb41d8963c949eeb767ee5f78289c54 P 38fc7605759a46d6b9e67bfb0d13372c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bbf31860ef14a1c8b7fed6f6a415153" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43389 } } peers { permanent_uuid: "38fc7605759a46d6b9e67bfb0d13372c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45793 } } peers { permanent_uuid: "7ce698571e9f4da3a09c77e6ad265cd2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39351 } }
I20260808 06:07:18.389496 25033 raft_consensus.cc:348] T 0fb41d8963c949eeb767ee5f78289c54 P 2bbf31860ef14a1c8b7fed6f6a415153 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bbf31860ef14a1c8b7fed6f6a415153" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43389 } } peers { permanent_uuid: "38fc7605759a46d6b9e67bfb0d13372c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45793 } } peers { permanent_uuid: "7ce698571e9f4da3a09c77e6ad265cd2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39351 } }
I20260808 06:07:18.389715 25036 raft_consensus.cc:374] T 0fb41d8963c949eeb767ee5f78289c54 P 38fc7605759a46d6b9e67bfb0d13372c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:07:18.389719 25033 raft_consensus.cc:374] T 0fb41d8963c949eeb767ee5f78289c54 P 2bbf31860ef14a1c8b7fed6f6a415153 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:07:18.389752 25036 raft_consensus.cc:729] T 0fb41d8963c949eeb767ee5f78289c54 P 38fc7605759a46d6b9e67bfb0d13372c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38fc7605759a46d6b9e67bfb0d13372c, State: Initialized, Role: FOLLOWER
I20260808 06:07:18.389779 25033 raft_consensus.cc:729] T 0fb41d8963c949eeb767ee5f78289c54 P 2bbf31860ef14a1c8b7fed6f6a415153 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bbf31860ef14a1c8b7fed6f6a415153, State: Initialized, Role: FOLLOWER
I20260808 06:07:18.390190 25036 consensus_queue.cc:260] T 0fb41d8963c949eeb767ee5f78289c54 P 38fc7605759a46d6b9e67bfb0d13372c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bbf31860ef14a1c8b7fed6f6a415153" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43389 } } peers { permanent_uuid: "38fc7605759a46d6b9e67bfb0d13372c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45793 } } peers { permanent_uuid: "7ce698571e9f4da3a09c77e6ad265cd2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39351 } }
I20260808 06:07:18.390200 25033 consensus_queue.cc:260] T 0fb41d8963c949eeb767ee5f78289c54 P 2bbf31860ef14a1c8b7fed6f6a415153 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bbf31860ef14a1c8b7fed6f6a415153" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43389 } } peers { permanent_uuid: "38fc7605759a46d6b9e67bfb0d13372c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45793 } } peers { permanent_uuid: "7ce698571e9f4da3a09c77e6ad265cd2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39351 } }
I20260808 06:07:18.390829 24805 heartbeater.cc:503] Master 127.1.151.190:39611 was elected leader, sending a full tablet report...
I20260808 06:07:18.391083 25015 heartbeater.cc:503] Master 127.1.151.190:39611 was elected leader, sending a full tablet report...
I20260808 06:07:18.391135 25033 ts_tablet_manager.cc:1428] T 0fb41d8963c949eeb767ee5f78289c54 P 2bbf31860ef14a1c8b7fed6f6a415153: Time spent starting tablet: real 0.004s user 0.002s sys 0.002s
I20260808 06:07:18.391135 25036 ts_tablet_manager.cc:1428] T 0fb41d8963c949eeb767ee5f78289c54 P 38fc7605759a46d6b9e67bfb0d13372c: Time spent starting tablet: real 0.004s user 0.002s sys 0.002s
W20260808 06:07:18.398216 24806 tablet.cc:2369] T 0fb41d8963c949eeb767ee5f78289c54 P 38fc7605759a46d6b9e67bfb0d13372c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 06:07:18.430872 25039 raft_consensus.cc:482] T 0fb41d8963c949eeb767ee5f78289c54 P 38fc7605759a46d6b9e67bfb0d13372c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:07:18.431049 25039 raft_consensus.cc:504] T 0fb41d8963c949eeb767ee5f78289c54 P 38fc7605759a46d6b9e67bfb0d13372c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bbf31860ef14a1c8b7fed6f6a415153" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43389 } } peers { permanent_uuid: "38fc7605759a46d6b9e67bfb0d13372c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45793 } } peers { permanent_uuid: "7ce698571e9f4da3a09c77e6ad265cd2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39351 } }
I20260808 06:07:18.432128 25039 leader_election.cc:288] T 0fb41d8963c949eeb767ee5f78289c54 P 38fc7605759a46d6b9e67bfb0d13372c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2bbf31860ef14a1c8b7fed6f6a415153 (127.1.151.131:43389), 7ce698571e9f4da3a09c77e6ad265cd2 (127.1.151.129:39351)
I20260808 06:07:18.436008 24807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:18.432359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:43389 (local address 127.1.151.130:38193)
0808 06:07:18.432503 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:18.432521 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:18.432629 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:18.432894 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:18.432898 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:18.432921 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:07:18.433056 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:18.433065 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:18.433949 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:18.433952 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:18.435052 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:18.435063 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:18.435692 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:18.435695 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:18.435774 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:18.435794 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 06:07:18.435850 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 06:07:18.435905 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 06:07:18.436069 25029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:18.432492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:38193 (local address 127.1.151.131:43389)
0808 06:07:18.432638 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:18.432643 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:18.432661 (+ 18us) server_negotiation.cc:407] Connection header received
0808 06:07:18.432737 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:18.432739 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:18.432790 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:18.432864 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:07:18.433187 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:18.433843 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:18.435183 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:18.435588 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:18.435703 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:18.435755 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:18.435881 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 06:07:18.435945 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 06:07:18.436740 25042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:18.432624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:39351 (local address 127.1.151.130:46079)
0808 06:07:18.433060 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:18.433072 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:18.433163 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:18.433446 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:18.433449 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:18.433467 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:07:18.433628 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:18.433636 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:18.434445 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:18.434448 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:18.435480 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:18.435488 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:18.436396 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:18.436400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:18.436490 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:18.436515 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 06:07:18.436580 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 06:07:18.436634 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":13,"thread_start_us":152,"threads_started":1}
I20260808 06:07:18.436820 24587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:18.432742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:46079 (local address 127.1.151.129:39351)
0808 06:07:18.432867 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:18.432871 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:18.433156 (+ 285us) server_negotiation.cc:407] Connection header received
0808 06:07:18.433245 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:18.433248 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:18.433316 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:18.433415 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:07:18.433738 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:18.434343 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:18.435597 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:18.436243 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:18.436364 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:18.436496 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:18.436627 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 06:07:18.436708 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 06:07:18.436834 24940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fb41d8963c949eeb767ee5f78289c54" candidate_uuid: "38fc7605759a46d6b9e67bfb0d13372c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bbf31860ef14a1c8b7fed6f6a415153" is_pre_election: true
I20260808 06:07:18.437151 24940 raft_consensus.cc:2454] T 0fb41d8963c949eeb767ee5f78289c54 P 2bbf31860ef14a1c8b7fed6f6a415153 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38fc7605759a46d6b9e67bfb0d13372c in term 0.
I20260808 06:07:18.437481 24500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fb41d8963c949eeb767ee5f78289c54" candidate_uuid: "38fc7605759a46d6b9e67bfb0d13372c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ce698571e9f4da3a09c77e6ad265cd2" is_pre_election: true
I20260808 06:07:18.437623 24619 leader_election.cc:302] T 0fb41d8963c949eeb767ee5f78289c54 P 38fc7605759a46d6b9e67bfb0d13372c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bbf31860ef14a1c8b7fed6f6a415153, 38fc7605759a46d6b9e67bfb0d13372c; no voters:
I20260808 06:07:18.437744 24500 raft_consensus.cc:2454] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38fc7605759a46d6b9e67bfb0d13372c in term 0.
I20260808 06:07:18.437939 25039 raft_consensus.cc:2781] T 0fb41d8963c949eeb767ee5f78289c54 P 38fc7605759a46d6b9e67bfb0d13372c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 06:07:18.438040 25039 raft_consensus.cc:482] T 0fb41d8963c949eeb767ee5f78289c54 P 38fc7605759a46d6b9e67bfb0d13372c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 06:07:18.438094 25039 raft_consensus.cc:3037] T 0fb41d8963c949eeb767ee5f78289c54 P 38fc7605759a46d6b9e67bfb0d13372c [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:07:18.439172 25039 raft_consensus.cc:504] T 0fb41d8963c949eeb767ee5f78289c54 P 38fc7605759a46d6b9e67bfb0d13372c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bbf31860ef14a1c8b7fed6f6a415153" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43389 } } peers { permanent_uuid: "38fc7605759a46d6b9e67bfb0d13372c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45793 } } peers { permanent_uuid: "7ce698571e9f4da3a09c77e6ad265cd2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39351 } }
I20260808 06:07:18.439466 25039 leader_election.cc:288] T 0fb41d8963c949eeb767ee5f78289c54 P 38fc7605759a46d6b9e67bfb0d13372c [CANDIDATE]: Term 1 election: Requested vote from peers 2bbf31860ef14a1c8b7fed6f6a415153 (127.1.151.131:43389), 7ce698571e9f4da3a09c77e6ad265cd2 (127.1.151.129:39351)
I20260808 06:07:18.439802 24940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fb41d8963c949eeb767ee5f78289c54" candidate_uuid: "38fc7605759a46d6b9e67bfb0d13372c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bbf31860ef14a1c8b7fed6f6a415153"
I20260808 06:07:18.439829 24500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fb41d8963c949eeb767ee5f78289c54" candidate_uuid: "38fc7605759a46d6b9e67bfb0d13372c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ce698571e9f4da3a09c77e6ad265cd2"
I20260808 06:07:18.439934 24500 raft_consensus.cc:3037] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:07:18.439929 24940 raft_consensus.cc:3037] T 0fb41d8963c949eeb767ee5f78289c54 P 2bbf31860ef14a1c8b7fed6f6a415153 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:07:18.441180 24940 raft_consensus.cc:2454] T 0fb41d8963c949eeb767ee5f78289c54 P 2bbf31860ef14a1c8b7fed6f6a415153 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38fc7605759a46d6b9e67bfb0d13372c in term 1.
I20260808 06:07:18.441179 24500 raft_consensus.cc:2454] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38fc7605759a46d6b9e67bfb0d13372c in term 1.
I20260808 06:07:18.441439 24619 leader_election.cc:302] T 0fb41d8963c949eeb767ee5f78289c54 P 38fc7605759a46d6b9e67bfb0d13372c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bbf31860ef14a1c8b7fed6f6a415153, 38fc7605759a46d6b9e67bfb0d13372c; no voters:
I20260808 06:07:18.441592 25039 raft_consensus.cc:2781] T 0fb41d8963c949eeb767ee5f78289c54 P 38fc7605759a46d6b9e67bfb0d13372c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:07:18.441800 25039 raft_consensus.cc:686] T 0fb41d8963c949eeb767ee5f78289c54 P 38fc7605759a46d6b9e67bfb0d13372c [term 1 LEADER]: Becoming Leader. State: Replica: 38fc7605759a46d6b9e67bfb0d13372c, State: Running, Role: LEADER
I20260808 06:07:18.442092 25039 consensus_queue.cc:237] T 0fb41d8963c949eeb767ee5f78289c54 P 38fc7605759a46d6b9e67bfb0d13372c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bbf31860ef14a1c8b7fed6f6a415153" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43389 } } peers { permanent_uuid: "38fc7605759a46d6b9e67bfb0d13372c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45793 } } peers { permanent_uuid: "7ce698571e9f4da3a09c77e6ad265cd2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39351 } }
I20260808 06:07:18.445331 24274 catalog_manager.cc:5495] T 0fb41d8963c949eeb767ee5f78289c54 P 38fc7605759a46d6b9e67bfb0d13372c reported cstate change: term changed from 0 to 1, leader changed from <none> to 38fc7605759a46d6b9e67bfb0d13372c (127.1.151.130). New cstate: current_term: 1 leader_uuid: "38fc7605759a46d6b9e67bfb0d13372c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bbf31860ef14a1c8b7fed6f6a415153" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38fc7605759a46d6b9e67bfb0d13372c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ce698571e9f4da3a09c77e6ad265cd2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39351 } health_report { overall_health: UNKNOWN } } }
W20260808 06:07:18.461267 24572 tablet.cc:2369] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 06:07:18.470265 25031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:18.467261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46380 (local address 127.1.151.130:45793)
0808 06:07:18.467372 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:18.467375 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:18.467388 (+ 13us) server_negotiation.cc:407] Connection header received
0808 06:07:18.467454 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:18.467456 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:18.467508 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:18.467590 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:07:18.467783 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:18.468257 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:18.469046 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:18.469264 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:18.469339 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:18.469548 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:07:18.469941 (+ 393us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:07:18.470024 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:18.470068 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 06:07:18.470116 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 06:07:18.475765 24500 raft_consensus.cc:1261] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 1 FOLLOWER]: Refusing update from remote peer 38fc7605759a46d6b9e67bfb0d13372c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:07:18.475770 24940 raft_consensus.cc:1261] T 0fb41d8963c949eeb767ee5f78289c54 P 2bbf31860ef14a1c8b7fed6f6a415153 [term 1 FOLLOWER]: Refusing update from remote peer 38fc7605759a46d6b9e67bfb0d13372c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:07:18.476420 25039 consensus_queue.cc:1035] T 0fb41d8963c949eeb767ee5f78289c54 P 38fc7605759a46d6b9e67bfb0d13372c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bbf31860ef14a1c8b7fed6f6a415153" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:07:18.476631 25043 consensus_queue.cc:1035] T 0fb41d8963c949eeb767ee5f78289c54 P 38fc7605759a46d6b9e67bfb0d13372c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ce698571e9f4da3a09c77e6ad265cd2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:07:18.482998 25050 mvcc.cc:205] Tried to move back new op lower bound from 7316149200789467136 to 7316149200661499904. Current Snapshot: MvccSnapshot[applied={T|T < 7316149200789467136}]
I20260808 06:07:18.491236 24275 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:36416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:07:18.491371 24275 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:36416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:07:18.493525 24275 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6: Sending DeleteTablet for 3 replicas of tablet 0fb41d8963c949eeb767ee5f78289c54
I20260808 06:07:18.494356 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 24339
I20260808 06:07:18.494328 24690 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fb41d8963c949eeb767ee5f78289c54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:07:18 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:46374
I20260808 06:07:18.494415 24907 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fb41d8963c949eeb767ee5f78289c54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:07:18 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:46760
I20260808 06:07:18.502276 25063 tablet_replica.cc:330] T 0fb41d8963c949eeb767ee5f78289c54 P 38fc7605759a46d6b9e67bfb0d13372c: stopping tablet replica
I20260808 06:07:18.502573 25063 raft_consensus.cc:2229] T 0fb41d8963c949eeb767ee5f78289c54 P 38fc7605759a46d6b9e67bfb0d13372c [term 1 LEADER]: Raft consensus shutting down.
I20260808 06:07:18.502934 25063 raft_consensus.cc:2258] T 0fb41d8963c949eeb767ee5f78289c54 P 38fc7605759a46d6b9e67bfb0d13372c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:07:18.507778 25063 ts_tablet_manager.cc:1905] T 0fb41d8963c949eeb767ee5f78289c54 P 38fc7605759a46d6b9e67bfb0d13372c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:07:18.509466 25064 tablet_replica.cc:330] T 0fb41d8963c949eeb767ee5f78289c54 P 2bbf31860ef14a1c8b7fed6f6a415153: stopping tablet replica
I20260808 06:07:18.509788 25064 raft_consensus.cc:2229] T 0fb41d8963c949eeb767ee5f78289c54 P 2bbf31860ef14a1c8b7fed6f6a415153 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:07:18.510115 25064 raft_consensus.cc:2258] T 0fb41d8963c949eeb767ee5f78289c54 P 2bbf31860ef14a1c8b7fed6f6a415153 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:07:18.511210 25063 ts_tablet_manager.cc:1918] T 0fb41d8963c949eeb767ee5f78289c54 P 38fc7605759a46d6b9e67bfb0d13372c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:07:18.511315 25063 log.cc:1196] T 0fb41d8963c949eeb767ee5f78289c54 P 38fc7605759a46d6b9e67bfb0d13372c: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/wal/wals/0fb41d8963c949eeb767ee5f78289c54
I20260808 06:07:18.511631 25063 ts_tablet_manager.cc:1939] T 0fb41d8963c949eeb767ee5f78289c54 P 38fc7605759a46d6b9e67bfb0d13372c: Deleting consensus metadata
I20260808 06:07:18.512403 25064 ts_tablet_manager.cc:1905] T 0fb41d8963c949eeb767ee5f78289c54 P 2bbf31860ef14a1c8b7fed6f6a415153: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:07:18.512823 24259 catalog_manager.cc:4841] TS 38fc7605759a46d6b9e67bfb0d13372c (127.1.151.130:45793): tablet 0fb41d8963c949eeb767ee5f78289c54 (table test-table [id=f7fa43f796c14c52937401ec150c5942]) successfully deleted
I20260808 06:07:18.515003 25064 ts_tablet_manager.cc:1918] T 0fb41d8963c949eeb767ee5f78289c54 P 2bbf31860ef14a1c8b7fed6f6a415153: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:07:18.515120 25064 log.cc:1196] T 0fb41d8963c949eeb767ee5f78289c54 P 2bbf31860ef14a1c8b7fed6f6a415153: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/wal/wals/0fb41d8963c949eeb767ee5f78289c54
I20260808 06:07:18.515452 25064 ts_tablet_manager.cc:1939] T 0fb41d8963c949eeb767ee5f78289c54 P 2bbf31860ef14a1c8b7fed6f6a415153: Deleting consensus metadata
I20260808 06:07:18.516618 24258 catalog_manager.cc:4841] TS 2bbf31860ef14a1c8b7fed6f6a415153 (127.1.151.131:43389): tablet 0fb41d8963c949eeb767ee5f78289c54 (table test-table [id=f7fa43f796c14c52937401ec150c5942]) successfully deleted
W20260808 06:07:18.518234 24258 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.151.129:39351 (error 108)
I20260808 06:07:18.518800 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 24577
I20260808 06:07:18.520818 25030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:18.518977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:39351 (local address 127.0.0.1:57788)
0808 06:07:18.520621 (+ 1644us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:18.520715 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.129:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1550}
W20260808 06:07:18.523765 24258 catalog_manager.cc:4584] TS 7ce698571e9f4da3a09c77e6ad265cd2 (127.1.151.129:39351): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0fb41d8963c949eeb767ee5f78289c54: Network error: Client connection negotiation failed: client connection to 127.1.151.129:39351: connect: Connection refused (error 111)
I20260808 06:07:18.543046 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 24809
I20260808 06:07:18.553895 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 24227
I20260808 06:07:18.568665 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:39611
--webserver_interface=127.1.151.190
--webserver_port=41435
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:34817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:39611
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:07:18.677359 25066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:07:18.677626 25066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:07:18.677675 25066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:07:18.681665 25066 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:07:18.681718 25066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:07:18.681736 25066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:07:18.681751 25066 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:07:18.681766 25066 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:07:18.686896 25066 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:34817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:39611
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:39611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=41435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:07:18.687873 25066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:07:18.688844 25066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:07:18.697105 25082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:18.697111 25079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:18.697391 25066 server_base.cc:1034] running on GCE node
W20260808 06:07:18.697233 25076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:18.697923 25066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:07:18.698983 25066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:07:18.700148 25066 hybrid_clock.cc:648] HybridClock initialized: now 1786169238700137 us; error 29 us; skew 500 ppm
Aug 08 06:07:18 dist-test-slave-zj8q krb5kdc[23362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169238, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:07:18.702776 25066 init.cc:377] Logged in from keytab as kudu/127.1.151.190@KRBTEST.COM (short username kudu)
I20260808 06:07:18.703595 25066 webserver.cc:460] Webserver started at http://127.1.151.190:41435/ using document root <none> and password file <none>
I20260808 06:07:18.704085 25066 fs_manager.cc:359] Metadata directory not provided
I20260808 06:07:18.704140 25066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:07:18.707772 25066 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:07:18.709805 25094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:18.710929 25066 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:07:18.711035 25066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "51bb59562e294210867a4e6920f254b6"
format_stamp: "Formatted at 2026-08-08 06:07:16 on dist-test-slave-zj8q"
server_key: "040f1b8315b7842701fb291a6e279abb"
server_key_iv: "5a2b5e974e7e7606478bb0600ab4c863"
server_key_version: "encryptionkey@0"
I20260808 06:07:18.711581 25066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:07:18.721421 25066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:07:18.724236 25066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:07:18.724399 25066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:07:18.731878 25066 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:39611
I20260808 06:07:18.732123 25189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:39611 every 8 connection(s)
I20260808 06:07:18.732872 25066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:07:18.734685 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 25066
I20260808 06:07:18.735111 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.129:39351
--local_ip_for_outbound_sockets=127.1.151.129
--tserver_master_addrs=127.1.151.190:39611
--webserver_port=32961
--webserver_interface=127.1.151.129
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:34817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 06:07:18.737188 25190 sys_catalog.cc:263] Verifying existing consensus state
I20260808 06:07:18.738834 25190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6: Bootstrap starting.
I20260808 06:07:18.756564 25190 log.cc:824] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6: Log is configured to *not* fsync() on all Append() calls
I20260808 06:07:18.767822 25190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 06:07:18.768151 25190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6: Bootstrap complete.
I20260808 06:07:18.772035 25190 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51bb59562e294210867a4e6920f254b6" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39611 } }
I20260808 06:07:18.772413 25190 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51bb59562e294210867a4e6920f254b6, State: Initialized, Role: FOLLOWER
I20260808 06:07:18.772874 25190 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51bb59562e294210867a4e6920f254b6" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39611 } }
I20260808 06:07:18.772993 25190 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:07:18.773062 25190 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:07:18.773154 25190 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [term 2 FOLLOWER]: Advancing to term 3
I20260808 06:07:18.775000 25190 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51bb59562e294210867a4e6920f254b6" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39611 } }
I20260808 06:07:18.775307 25190 leader_election.cc:302] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51bb59562e294210867a4e6920f254b6; no voters:
I20260808 06:07:18.775597 25190 leader_election.cc:288] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 06:07:18.775736 25198 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 06:07:18.775941 25198 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [term 3 LEADER]: Becoming Leader. State: Replica: 51bb59562e294210867a4e6920f254b6, State: Running, Role: LEADER
I20260808 06:07:18.776235 25198 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51bb59562e294210867a4e6920f254b6" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39611 } }
I20260808 06:07:18.776818 25190 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:07:18.777500 25199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "51bb59562e294210867a4e6920f254b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51bb59562e294210867a4e6920f254b6" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39611 } } }
I20260808 06:07:18.777630 25199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [sys.catalog]: This master's current role is: LEADER
I20260808 06:07:18.777963 25200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51bb59562e294210867a4e6920f254b6. Latest consensus state: current_term: 3 leader_uuid: "51bb59562e294210867a4e6920f254b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51bb59562e294210867a4e6920f254b6" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39611 } } }
I20260808 06:07:18.778048 25200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51bb59562e294210867a4e6920f254b6 [sys.catalog]: This master's current role is: LEADER
I20260808 06:07:18.778525 25203 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:07:18.783524 25203 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:07:18.784183 25203 catalog_manager.cc:1259] Loaded cluster ID: 7bdd4ae20a4e4ac18482b0cdadc08061
I20260808 06:07:18.784243 25203 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 06:07:18.785383 25203 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fc129ff49ad google::LogMessage::Fail() at ??:0
@ 0x7fc129ff788a google::LogMessage::SendToLog() at ??:0
@ 0x7fc129ff44de google::LogMessage::Flush() at ??:0
@ 0x7fc129ff5a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fc134933fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fc134934d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fc134931579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fc134975640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fc12c2463d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fc12c246c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fc12c2483f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fc12c237512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fc12d87b6db start_thread at ??:0
@ 0x7fc128af261f clone at ??:0
W20260808 06:07:18.858937 25194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:07:18.859175 25194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:07:18.859252 25194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:07:18.863225 25194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:07:18.863277 25194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:07:18.863349 25194 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.129
I20260808 06:07:18.868469 25194 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:34817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.129:39351
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.151.129
--webserver_port=32961
--tserver_master_addrs=127.1.151.190:39611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.129
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:07:18.869459 25194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:07:18.870476 25194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:07:18.879874 25251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:18.879879 25250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:18.879999 25253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:18.880298 25194 server_base.cc:1034] running on GCE node
I20260808 06:07:18.880646 25194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:07:18.881228 25194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:07:18.882392 25194 hybrid_clock.cc:648] HybridClock initialized: now 1786169238882372 us; error 33 us; skew 500 ppm
Aug 08 06:07:18 dist-test-slave-zj8q krb5kdc[23362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169238, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:07:18.884909 25194 init.cc:377] Logged in from keytab as kudu/127.1.151.129@KRBTEST.COM (short username kudu)
I20260808 06:07:18.885931 25194 webserver.cc:460] Webserver started at http://127.1.151.129:32961/ using document root <none> and password file <none>
I20260808 06:07:18.886602 25194 fs_manager.cc:359] Metadata directory not provided
I20260808 06:07:18.886669 25194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:07:18.890345 25194 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 06:07:18.892375 25265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:18.893525 25194 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:07:18.893620 25194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/wal
uuid: "7ce698571e9f4da3a09c77e6ad265cd2"
format_stamp: "Formatted at 2026-08-08 06:07:16 on dist-test-slave-zj8q"
server_key: "dbff06773774fe574da4b5b669a147ba"
server_key_iv: "ffba471894e68b82de61319530746781"
server_key_version: "encryptionkey@0"
I20260808 06:07:18.894127 25194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:07:18.906029 25194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:07:18.908891 25194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:07:18.909049 25194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:07:18.909602 25194 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:07:18.910686 25276 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 06:07:18.914503 25194 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 06:07:18.914556 25194 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260808 06:07:18.914599 25194 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 06:07:18.916144 25194 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 06:07:18.916208 25194 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260808 06:07:18.916473 25276 tablet_bootstrap.cc:492] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2: Bootstrap starting.
I20260808 06:07:18.926486 25194 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.129:39351
I20260808 06:07:18.926775 25460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.129:39351 every 8 connection(s)
I20260808 06:07:18.927630 25194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
I20260808 06:07:18.932363 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 25194
I20260808 06:07:18.932824 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.130:45793
--local_ip_for_outbound_sockets=127.1.151.130
--tserver_master_addrs=127.1.151.190:39611
--webserver_port=33379
--webserver_interface=127.1.151.130
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:34817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 06:07:18.936679 25276 log.cc:824] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2: Log is configured to *not* fsync() on all Append() calls
I20260808 06:07:18.945706 25276 tablet_bootstrap.cc:492] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 06:07:18.946116 25276 tablet_bootstrap.cc:492] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2: Bootstrap complete.
I20260808 06:07:18.947608 25276 ts_tablet_manager.cc:1397] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2: Time spent bootstrapping tablet: real 0.031s user 0.018s sys 0.011s
I20260808 06:07:18.950687 25276 raft_consensus.cc:348] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bbf31860ef14a1c8b7fed6f6a415153" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43389 } } peers { permanent_uuid: "38fc7605759a46d6b9e67bfb0d13372c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45793 } } peers { permanent_uuid: "7ce698571e9f4da3a09c77e6ad265cd2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39351 } }
I20260808 06:07:18.951201 25276 raft_consensus.cc:729] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ce698571e9f4da3a09c77e6ad265cd2, State: Initialized, Role: FOLLOWER
I20260808 06:07:18.951709 25276 consensus_queue.cc:260] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bbf31860ef14a1c8b7fed6f6a415153" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43389 } } peers { permanent_uuid: "38fc7605759a46d6b9e67bfb0d13372c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45793 } } peers { permanent_uuid: "7ce698571e9f4da3a09c77e6ad265cd2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39351 } }
I20260808 06:07:18.952687 25276 ts_tablet_manager.cc:1428] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
W20260808 06:07:18.999258 25466 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:07:18.929861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.129:56953)
0808 06:07:18.930443 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:18.930502 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:18.931523 (+ 1021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:18.998363 (+ 66840us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":152,"threads_started":1}
W20260808 06:07:18.999842 25461 heartbeater.cc:644] Failed to heartbeat to 127.1.151.190:39611 (0 consecutive failures): Network error: Failed to ping master at 127.1.151.190:39611: Client connection negotiation failed: client connection to 127.1.151.190:39611: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 06:07:19.000607 25466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:19.000349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.129:57215)
0808 06:07:19.000479 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:19.000549 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 06:07:19.001319 25466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:19.001103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.129:36407)
0808 06:07:19.001210 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:19.001261 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 06:07:19.001477 25461 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 06:07:19.052805 25468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:07:19.053042 25468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:07:19.053098 25468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:07:19.056975 25468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:07:19.057029 25468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:07:19.057106 25468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.130
I20260808 06:07:19.062222 25468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:34817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.130:45793
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.151.130
--webserver_port=33379
--tserver_master_addrs=127.1.151.190:39611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.130
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:07:19.063236 25468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:07:19.064229 25468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:07:19.073344 25484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:19.073366 25481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:19.073380 25479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:19.073475 25468 server_base.cc:1034] running on GCE node
I20260808 06:07:19.073889 25468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:07:19.074491 25468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:07:19.075667 25468 hybrid_clock.cc:648] HybridClock initialized: now 1786169239075649 us; error 34 us; skew 500 ppm
Aug 08 06:07:19 dist-test-slave-zj8q krb5kdc[23362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169239, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:07:19.078475 25468 init.cc:377] Logged in from keytab as kudu/127.1.151.130@KRBTEST.COM (short username kudu)
I20260808 06:07:19.079329 25468 webserver.cc:460] Webserver started at http://127.1.151.130:33379/ using document root <none> and password file <none>
I20260808 06:07:19.079878 25468 fs_manager.cc:359] Metadata directory not provided
I20260808 06:07:19.079941 25468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:07:19.083604 25468 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:07:19.085543 25494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:19.086740 25468 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:07:19.086850 25468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/wal
uuid: "38fc7605759a46d6b9e67bfb0d13372c"
format_stamp: "Formatted at 2026-08-08 06:07:16 on dist-test-slave-zj8q"
server_key: "47e94270c951564af4414842d69126ca"
server_key_iv: "d47dd0cb305710dbfdacea7d7112ae8f"
server_key_version: "encryptionkey@0"
I20260808 06:07:19.087352 25468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:07:19.103720 25468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:07:19.106596 25468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:07:19.106765 25468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:07:19.107311 25468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:07:19.108085 25468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:07:19.108134 25468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:19.108183 25468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:07:19.108208 25468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:19.116928 25468 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.130:45793
I20260808 06:07:19.116953 25664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.130:45793 every 8 connection(s)
I20260808 06:07:19.117880 25468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
I20260808 06:07:19.121317 25669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:19.119887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.130:46639)
0808 06:07:19.120527 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:19.120628 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":113,"threads_started":1}
W20260808 06:07:19.121706 25667 heartbeater.cc:644] Failed to heartbeat to 127.1.151.190:39611 (0 consecutive failures): Network error: Failed to ping master at 127.1.151.190:39611: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
I20260808 06:07:19.122265 25669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:19.122020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.130:48669)
0808 06:07:19.122106 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:19.122180 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 06:07:19.122485 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 25468
I20260808 06:07:19.122937 25669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:19.122719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.130:41215)
0808 06:07:19.122825 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:19.122876 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 06:07:19.122910 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.131:43389
--local_ip_for_outbound_sockets=127.1.151.131
--tserver_master_addrs=127.1.151.190:39611
--webserver_port=33361
--webserver_interface=127.1.151.131
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:34817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:07:19.123101 25667 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 06:07:19.235702 25671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:07:19.235955 25671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:07:19.236013 25671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:07:19.240087 25671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:07:19.240149 25671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:07:19.240228 25671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.131
I20260808 06:07:19.245587 25671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:34817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.131:43389
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.151.131
--webserver_port=33361
--tserver_master_addrs=127.1.151.190:39611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.131
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:07:19.246654 25671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:07:19.247637 25671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:07:19.257126 25684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:19.257167 25679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:19.257228 25671 server_base.cc:1034] running on GCE node
W20260808 06:07:19.257164 25682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:19.257784 25671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:07:19.258431 25671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:07:19.259593 25671 hybrid_clock.cc:648] HybridClock initialized: now 1786169239259577 us; error 56 us; skew 500 ppm
Aug 08 06:07:19 dist-test-slave-zj8q krb5kdc[23362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169239, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:07:19.262245 25671 init.cc:377] Logged in from keytab as kudu/127.1.151.131@KRBTEST.COM (short username kudu)
I20260808 06:07:19.263146 25671 webserver.cc:460] Webserver started at http://127.1.151.131:33361/ using document root <none> and password file <none>
I20260808 06:07:19.263710 25671 fs_manager.cc:359] Metadata directory not provided
I20260808 06:07:19.263777 25671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:07:19.267812 25671 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.003s
I20260808 06:07:19.269894 25696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:19.271086 25671 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 06:07:19.271212 25671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/wal
uuid: "2bbf31860ef14a1c8b7fed6f6a415153"
format_stamp: "Formatted at 2026-08-08 06:07:17 on dist-test-slave-zj8q"
server_key: "8180099d00e94bae7912f5617beffd41"
server_key_iv: "cc4a4e57cb71a2be30cfc51900df39be"
server_key_version: "encryptionkey@0"
I20260808 06:07:19.271727 25671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:07:19.301203 25671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:07:19.304307 25671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:07:19.304488 25671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:07:19.305132 25671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:07:19.305994 25671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:07:19.306046 25671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:19.306098 25671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:07:19.306124 25671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:19.314760 25671 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.131:43389
I20260808 06:07:19.314841 25902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.131:43389 every 8 connection(s)
I20260808 06:07:19.315733 25671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
I20260808 06:07:19.319326 25908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:19.317838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.131:59831)
0808 06:07:19.318539 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:19.318654 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":123,"threads_started":1}
W20260808 06:07:19.319813 25905 heartbeater.cc:644] Failed to heartbeat to 127.1.151.190:39611 (0 consecutive failures): Network error: Failed to ping master at 127.1.151.190:39611: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
I20260808 06:07:19.320245 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 25671
I20260808 06:07:19.320451 25908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:19.320208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.131:42815)
0808 06:07:19.320328 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:19.320392 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 06:07:19.321151 25908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:19.320912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.131:39813)
0808 06:07:19.321025 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:19.321086 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 06:07:19.321324 25905 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 06:07:20.002511 25910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:20.002031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.129:46549)
0808 06:07:20.002336 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:20.002392 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":80,"threads_started":1}
I20260808 06:07:20.123934 25911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:20.123548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.130:43843)
0808 06:07:20.123784 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:20.123835 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":67,"threads_started":1}
I20260808 06:07:20.322257 25915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:20.321846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.131:38731)
0808 06:07:20.322064 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:20.322125 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":69,"threads_started":1}
I20260808 06:07:20.498543 25917 raft_consensus.cc:482] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:07:20.498785 25917 raft_consensus.cc:504] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bbf31860ef14a1c8b7fed6f6a415153" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43389 } } peers { permanent_uuid: "38fc7605759a46d6b9e67bfb0d13372c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45793 } } peers { permanent_uuid: "7ce698571e9f4da3a09c77e6ad265cd2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39351 } }
I20260808 06:07:20.499861 25917 leader_election.cc:288] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2bbf31860ef14a1c8b7fed6f6a415153 (127.1.151.131:43389), 38fc7605759a46d6b9e67bfb0d13372c (127.1.151.130:45793)
Aug 08 06:07:20 dist-test-slave-zj8q krb5kdc[23362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169238, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for kudu/127.1.151.131@KRBTEST.COM
Aug 08 06:07:20 dist-test-slave-zj8q krb5kdc[23362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169238, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for kudu/127.1.151.130@KRBTEST.COM
I20260808 06:07:20.512699 25920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:20.500132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:43389 (local address 127.1.151.129:35909)
0808 06:07:20.500457 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:20.500471 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:20.500568 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:20.502148 (+ 1580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:20.502181 (+ 33us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:20.502727 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:07:20.503209 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:20.503223 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:20.504480 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:20.504483 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:20.505479 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:20.505490 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:20.506197 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:20.506201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:20.506424 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:20.507075 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:07:20.507095 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:07:20.508483 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:07:20.510479 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:20.510485 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:20.510499 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:20.510750 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:20.511069 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:20.511073 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:20.511075 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:20.511209 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:20.511788 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:07:20.511794 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:07:20.512000 (+ 206us) client_negotiation.cc:769] Sending connection context
0808 06:07:20.512160 (+ 160us) client_negotiation.cc:240] Negotiation successful
0808 06:07:20.512497 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":156,"spinlock_wait_cycles":26496,"thread_start_us":79,"threads_started":1}
I20260808 06:07:20.512720 25923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:20.500315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:35909 (local address 127.1.151.131:43389)
0808 06:07:20.500669 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:20.500693 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:20.500732 (+ 39us) server_negotiation.cc:407] Connection header received
0808 06:07:20.501438 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:20.501444 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:20.501789 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:20.502029 (+ 240us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:20.503376 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:20.504335 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:20.505622 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:20.506058 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:20.506220 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:20.508683 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:20.508688 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:20.508699 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:20.508727 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:20.510366 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:20.510856 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:20.510861 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:20.510865 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:20.510979 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:20.511296 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:20.511300 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:20.511301 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:20.511673 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:20.511792 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:20.512298 (+ 506us) server_negotiation.cc:299] Negotiation successful
0808 06:07:20.512536 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":73,"threads_started":1}
I20260808 06:07:20.512729 25924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:20.500358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:45583 (local address 127.1.151.130:45793)
0808 06:07:20.500799 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:20.500820 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:20.500852 (+ 32us) server_negotiation.cc:407] Connection header received
0808 06:07:20.501636 (+ 784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:20.501644 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:20.501953 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:20.502310 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:20.503370 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:20.504339 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:20.505605 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:20.506069 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:20.506249 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:20.509008 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:20.509015 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:20.509029 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:20.509063 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:20.510661 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:20.511056 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:20.511060 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:20.511064 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:20.511171 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:20.511421 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:20.511424 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:20.511426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:20.511775 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:20.511888 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:20.512290 (+ 402us) server_negotiation.cc:299] Negotiation successful
0808 06:07:20.512530 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":106,"threads_started":1}
I20260808 06:07:20.512717 25910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:20.500106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:45793 (local address 127.1.151.129:45583)
0808 06:07:20.500268 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:20.500284 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:20.500391 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:20.502249 (+ 1858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:20.502252 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:20.502712 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:07:20.503209 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:20.503223 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:20.504479 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:20.504482 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:20.505477 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:20.505488 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:20.506214 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:20.506218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:20.506424 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:20.507199 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:07:20.507211 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:07:20.508826 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:07:20.510766 (+ 1940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:20.510769 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:20.510771 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:20.510952 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:20.511262 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:20.511265 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:20.511267 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:20.511342 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:20.511877 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:07:20.511880 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:07:20.512000 (+ 120us) client_negotiation.cc:769] Sending connection context
0808 06:07:20.512160 (+ 160us) client_negotiation.cc:240] Negotiation successful
0808 06:07:20.512570 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"mutex_wait_us":310,"spinlock_wait_cycles":7040}
W20260808 06:07:20.515475 25273 leader_election.cc:341] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 38fc7605759a46d6b9e67bfb0d13372c (127.1.151.130:45793): Not found: Tablet not found: 0fb41d8963c949eeb767ee5f78289c54
W20260808 06:07:20.515609 25272 leader_election.cc:341] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2bbf31860ef14a1c8b7fed6f6a415153 (127.1.151.131:43389): Not found: Tablet not found: 0fb41d8963c949eeb767ee5f78289c54
I20260808 06:07:20.515702 25272 leader_election.cc:302] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ce698571e9f4da3a09c77e6ad265cd2; no voters: 2bbf31860ef14a1c8b7fed6f6a415153, 38fc7605759a46d6b9e67bfb0d13372c
I20260808 06:07:20.515951 25917 raft_consensus.cc:2726] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 06:07:21.003446 25910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:21.003190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.129:52131)
0808 06:07:21.003302 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:21.003358 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 06:07:21.124939 25926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:21.124527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.130:58815)
0808 06:07:21.124797 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:21.124852 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":80,"threads_started":1}
I20260808 06:07:21.323292 25927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:21.322898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.131:54887)
0808 06:07:21.323147 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:21.323197 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":73,"threads_started":1}
I20260808 06:07:22.004551 25928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:22.004148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.129:49487)
0808 06:07:22.004395 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:22.004450 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":79,"threads_started":1}
I20260808 06:07:22.126114 25929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:22.125728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.130:42123)
0808 06:07:22.125961 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:22.126013 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":71,"threads_started":1}
I20260808 06:07:22.324349 25931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:22.323952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.131:38859)
0808 06:07:22.324186 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:22.324237 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":75,"threads_started":1}
I20260808 06:07:22.645680 25932 raft_consensus.cc:482] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:07:22.645812 25932 raft_consensus.cc:504] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bbf31860ef14a1c8b7fed6f6a415153" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43389 } } peers { permanent_uuid: "38fc7605759a46d6b9e67bfb0d13372c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45793 } } peers { permanent_uuid: "7ce698571e9f4da3a09c77e6ad265cd2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39351 } }
I20260808 06:07:22.646202 25932 leader_election.cc:288] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2bbf31860ef14a1c8b7fed6f6a415153 (127.1.151.131:43389), 38fc7605759a46d6b9e67bfb0d13372c (127.1.151.130:45793)
W20260808 06:07:22.647084 25273 leader_election.cc:341] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 38fc7605759a46d6b9e67bfb0d13372c (127.1.151.130:45793): Not found: Tablet not found: 0fb41d8963c949eeb767ee5f78289c54
W20260808 06:07:22.647207 25272 leader_election.cc:341] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2bbf31860ef14a1c8b7fed6f6a415153 (127.1.151.131:43389): Not found: Tablet not found: 0fb41d8963c949eeb767ee5f78289c54
I20260808 06:07:22.647290 25272 leader_election.cc:302] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ce698571e9f4da3a09c77e6ad265cd2; no voters: 2bbf31860ef14a1c8b7fed6f6a415153, 38fc7605759a46d6b9e67bfb0d13372c
I20260808 06:07:22.647423 25932 raft_consensus.cc:2726] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 06:07:23.005831 25933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:23.005296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.129:43107)
0808 06:07:23.005647 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:23.005712 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":97,"threads_started":1}
I20260808 06:07:23.127247 25934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:23.126824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.130:34485)
0808 06:07:23.127096 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:23.127154 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":74,"threads_started":1}
I20260808 06:07:23.325402 25935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:23.325010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.131:44793)
0808 06:07:23.325257 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:23.325313 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":77,"threads_started":1}
I20260808 06:07:24.007193 25939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:24.006720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.129:60473)
0808 06:07:24.007015 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:24.007072 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":74,"threads_started":1}
I20260808 06:07:24.128324 25940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:24.127949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.130:58593)
0808 06:07:24.128180 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:24.128233 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":77,"threads_started":1}
I20260808 06:07:24.326574 25941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:24.326185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.131:49363)
0808 06:07:24.326424 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:24.326481 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":66,"threads_started":1}
I20260808 06:07:24.748436 25943 raft_consensus.cc:482] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:07:24.748584 25943 raft_consensus.cc:504] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bbf31860ef14a1c8b7fed6f6a415153" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43389 } } peers { permanent_uuid: "38fc7605759a46d6b9e67bfb0d13372c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45793 } } peers { permanent_uuid: "7ce698571e9f4da3a09c77e6ad265cd2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39351 } }
I20260808 06:07:24.749058 25943 leader_election.cc:288] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2bbf31860ef14a1c8b7fed6f6a415153 (127.1.151.131:43389), 38fc7605759a46d6b9e67bfb0d13372c (127.1.151.130:45793)
W20260808 06:07:24.749981 25273 leader_election.cc:341] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 38fc7605759a46d6b9e67bfb0d13372c (127.1.151.130:45793): Not found: Tablet not found: 0fb41d8963c949eeb767ee5f78289c54
W20260808 06:07:24.750129 25272 leader_election.cc:341] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2bbf31860ef14a1c8b7fed6f6a415153 (127.1.151.131:43389): Not found: Tablet not found: 0fb41d8963c949eeb767ee5f78289c54
I20260808 06:07:24.750212 25272 leader_election.cc:302] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ce698571e9f4da3a09c77e6ad265cd2; no voters: 2bbf31860ef14a1c8b7fed6f6a415153, 38fc7605759a46d6b9e67bfb0d13372c
I20260808 06:07:24.750388 25943 raft_consensus.cc:2726] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 06:07:25.008322 25944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:25.007906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.129:44469)
0808 06:07:25.008162 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:25.008223 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":73,"threads_started":1}
I20260808 06:07:25.129393 25945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:25.129019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.130:37689)
0808 06:07:25.129253 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:25.129305 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":80,"threads_started":1}
I20260808 06:07:25.327649 25946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:25.327237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.131:58367)
0808 06:07:25.327499 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:25.327557 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":74,"threads_started":1}
I20260808 06:07:26.009729 25947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:26.009217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.129:49159)
0808 06:07:26.009551 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:26.009622 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":87,"threads_started":1}
I20260808 06:07:26.130780 25950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:26.130237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.130:47477)
0808 06:07:26.130580 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:26.130683 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":75,"threads_started":1}
I20260808 06:07:26.328998 25951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:26.328460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.131:33255)
0808 06:07:26.328836 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:26.328902 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":115,"threads_started":1}
I20260808 06:07:27.011094 25952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:27.010647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.129:51183)
0808 06:07:27.010924 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:27.010992 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":84,"threads_started":1}
I20260808 06:07:27.132028 25953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:27.131581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.130:51361)
0808 06:07:27.131867 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:27.131934 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":81,"threads_started":1}
I20260808 06:07:27.330371 25954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:27.329870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.131:45691)
0808 06:07:27.330205 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:27.330272 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":98,"threads_started":1}
I20260808 06:07:28.012252 25955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:28.011834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.129:46797)
0808 06:07:28.012095 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:28.012153 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":98,"threads_started":1}
I20260808 06:07:28.133152 25956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:28.132740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.130:32851)
0808 06:07:28.132994 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:28.133050 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":82,"threads_started":1}
I20260808 06:07:28.331511 25957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:28.331066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.131:43479)
0808 06:07:28.331353 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:28.331413 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":98,"threads_started":1}
I20260808 06:07:28.756711 25959 raft_consensus.cc:482] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:07:28.756856 25959 raft_consensus.cc:504] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bbf31860ef14a1c8b7fed6f6a415153" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43389 } } peers { permanent_uuid: "38fc7605759a46d6b9e67bfb0d13372c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45793 } } peers { permanent_uuid: "7ce698571e9f4da3a09c77e6ad265cd2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39351 } }
I20260808 06:07:28.757283 25959 leader_election.cc:288] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2bbf31860ef14a1c8b7fed6f6a415153 (127.1.151.131:43389), 38fc7605759a46d6b9e67bfb0d13372c (127.1.151.130:45793)
W20260808 06:07:28.758107 25273 leader_election.cc:341] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 38fc7605759a46d6b9e67bfb0d13372c (127.1.151.130:45793): Not found: Tablet not found: 0fb41d8963c949eeb767ee5f78289c54
W20260808 06:07:28.758255 25272 leader_election.cc:341] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2bbf31860ef14a1c8b7fed6f6a415153 (127.1.151.131:43389): Not found: Tablet not found: 0fb41d8963c949eeb767ee5f78289c54
I20260808 06:07:28.758318 25272 leader_election.cc:302] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ce698571e9f4da3a09c77e6ad265cd2; no voters: 2bbf31860ef14a1c8b7fed6f6a415153, 38fc7605759a46d6b9e67bfb0d13372c
I20260808 06:07:28.758450 25959 raft_consensus.cc:2726] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 06:07:29.013479 25960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:29.013046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.129:50743)
0808 06:07:29.013316 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:29.013379 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":102,"threads_started":1}
I20260808 06:07:29.134382 25962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:29.133926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.130:59455)
0808 06:07:29.134228 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:29.134285 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":71,"threads_started":1}
I20260808 06:07:29.332608 25964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:29.332208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.131:51655)
0808 06:07:29.332456 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:29.332513 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":70,"threads_started":1}
I20260808 06:07:30.014878 25965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:30.014347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.129:47199)
0808 06:07:30.014677 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:30.014749 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":111,"threads_started":1}
I20260808 06:07:30.135535 25967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:30.135048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.130:37437)
0808 06:07:30.135368 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:30.135429 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":112,"threads_started":1}
I20260808 06:07:30.333734 25969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:30.333307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.131:53701)
0808 06:07:30.333581 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:30.333638 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":90,"threads_started":1}
I20260808 06:07:30.888849 25970 raft_consensus.cc:482] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:07:30.888988 25970 raft_consensus.cc:504] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bbf31860ef14a1c8b7fed6f6a415153" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 43389 } } peers { permanent_uuid: "38fc7605759a46d6b9e67bfb0d13372c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45793 } } peers { permanent_uuid: "7ce698571e9f4da3a09c77e6ad265cd2" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 39351 } }
I20260808 06:07:30.889421 25970 leader_election.cc:288] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2bbf31860ef14a1c8b7fed6f6a415153 (127.1.151.131:43389), 38fc7605759a46d6b9e67bfb0d13372c (127.1.151.130:45793)
W20260808 06:07:30.890189 25272 leader_election.cc:341] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2bbf31860ef14a1c8b7fed6f6a415153 (127.1.151.131:43389): Not found: Tablet not found: 0fb41d8963c949eeb767ee5f78289c54
W20260808 06:07:30.890311 25273 leader_election.cc:341] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 38fc7605759a46d6b9e67bfb0d13372c (127.1.151.130:45793): Not found: Tablet not found: 0fb41d8963c949eeb767ee5f78289c54
I20260808 06:07:30.890417 25273 leader_election.cc:302] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ce698571e9f4da3a09c77e6ad265cd2; no voters: 2bbf31860ef14a1c8b7fed6f6a415153, 38fc7605759a46d6b9e67bfb0d13372c
I20260808 06:07:30.890571 25970 raft_consensus.cc:2726] T 0fb41d8963c949eeb767ee5f78289c54 P 7ce698571e9f4da3a09c77e6ad265cd2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 06:07:31.016033 25971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:31.015611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.129:51203)
0808 06:07:31.015874 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:31.015933 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":94,"threads_started":1}
I20260808 06:07:31.136801 25972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:31.136361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.130:54291)
0808 06:07:31.136650 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:31.136705 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":95,"threads_started":1}
I20260808 06:07:31.334895 25973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:31.334459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.131:50371)
0808 06:07:31.334740 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:31.334800 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":70,"threads_started":1}
I20260808 06:07:32.017208 25975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:32.016758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.129:45629)
0808 06:07:32.017012 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:32.017080 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":97,"threads_started":1}
I20260808 06:07:32.137948 25976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:32.137540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.130:39057)
0808 06:07:32.137790 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:32.137847 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":94,"threads_started":1}
I20260808 06:07:32.336115 25978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:32.335669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.131:41283)
0808 06:07:32.335950 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:32.336009 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":83,"threads_started":1}
I20260808 06:07:33.018504 25980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:33.018019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.129:50681)
0808 06:07:33.018339 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:33.018399 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":126,"threads_started":1}
I20260808 06:07:33.139047 25981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:33.138646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.130:59215)
0808 06:07:33.138898 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:33.138953 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":79,"threads_started":1}
I20260808 06:07:33.337291 25982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:33.336852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.131:53559)
0808 06:07:33.337115 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:33.337180 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":81,"threads_started":1}
I20260808 06:07:34.020052 25984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:34.019550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.129:60659)
0808 06:07:34.019866 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:34.019931 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":93,"threads_started":1}
I20260808 06:07:34.140275 25985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:34.139886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.130:48113)
0808 06:07:34.140120 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:34.140177 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":71,"threads_started":1}
I20260808 06:07:34.324734 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 25194
I20260808 06:07:34.338764 25989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:34.338352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39611 (local address 127.1.151.131:56141)
0808 06:07:34.338604 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:34.338664 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":73,"threads_started":1}
I20260808 06:07:34.341713 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 25468
I20260808 06:07:34.355901 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 25671
2026-08-08T06:07:34Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19379 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:07:34 dist-test-slave-zj8q krb5kdc[25996](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:07:34 dist-test-slave-zj8q krb5kdc[25996](info): set up 2 sockets
Aug 08 06:07:34 dist-test-slave-zj8q krb5kdc[25996](info): commencing operation
krb5kdc: starting...
W20260808 06:07:35.506552 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:07:35 dist-test-slave-zj8q krb5kdc[25996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169255, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:07:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:07:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.151.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
I20260808 06:07:35.626639 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:39415
--webserver_interface=127.1.151.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:35001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:39415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:07:35.735003 26021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:07:35.735246 26021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:07:35.735294 26021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:07:35.739194 26021 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 06:07:35.739241 26021 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 06:07:35.739255 26021 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 06:07:35.739269 26021 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:07:35.739300 26021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:07:35.739331 26021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:07:35.739356 26021 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:07:35.739383 26021 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:07:35.744545 26021 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:35001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:39415
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:39415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:07:35.745556 26021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:07:35.746578 26021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 06:07:35.755013 26021 server_base.cc:1034] running on GCE node
W20260808 06:07:35.754979 26030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:35.755014 26031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:35.754978 26034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:35.755765 26021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:07:35.756837 26021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:07:35.758000 26021 hybrid_clock.cc:648] HybridClock initialized: now 1786169255757987 us; error 25 us; skew 500 ppm
Aug 08 06:07:35 dist-test-slave-zj8q krb5kdc[25996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169255, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:07:35.760649 26021 init.cc:377] Logged in from keytab as kudu/127.1.151.190@KRBTEST.COM (short username kudu)
I20260808 06:07:35.761478 26021 webserver.cc:460] Webserver started at http://127.1.151.190:41189/ using document root <none> and password file <none>
I20260808 06:07:35.761982 26021 fs_manager.cc:359] Metadata directory not provided
I20260808 06:07:35.762048 26021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:07:35.762298 26021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:07:35.764220 26021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "1d12e98c2324449b8fbf2a5e12396579"
format_stamp: "Formatted at 2026-08-08 06:07:35 on dist-test-slave-zj8q"
server_key: "c15b5a4cf93da8907c0fddcdc9eaa2ca"
server_key_iv: "8f3e1e1f4fe612d95af6b4a87993804d"
server_key_version: "encryptionkey@0"
I20260808 06:07:35.764745 26021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "1d12e98c2324449b8fbf2a5e12396579"
format_stamp: "Formatted at 2026-08-08 06:07:35 on dist-test-slave-zj8q"
server_key: "c15b5a4cf93da8907c0fddcdc9eaa2ca"
server_key_iv: "8f3e1e1f4fe612d95af6b4a87993804d"
server_key_version: "encryptionkey@0"
I20260808 06:07:35.768213 26021 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 06:07:35.770519 26051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:35.771929 26021 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:07:35.772040 26021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "1d12e98c2324449b8fbf2a5e12396579"
format_stamp: "Formatted at 2026-08-08 06:07:35 on dist-test-slave-zj8q"
server_key: "c15b5a4cf93da8907c0fddcdc9eaa2ca"
server_key_iv: "8f3e1e1f4fe612d95af6b4a87993804d"
server_key_version: "encryptionkey@0"
I20260808 06:07:35.772141 26021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:07:35.791947 26021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:07:35.795027 26021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:07:35.795213 26021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:07:35.802073 26021 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:39415
I20260808 06:07:35.802416 26136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:39415 every 8 connection(s)
I20260808 06:07:35.803063 26021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:07:35.805933 26140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:07:35.811720 26140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579: Bootstrap starting.
I20260808 06:07:35.812747 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 26021
I20260808 06:07:35.812867 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:07:35.813114 1630 external_mini_cluster.cc:1428] Setting key eb717066d31782ba5625f7e7e3c088e0
I20260808 06:07:35.814558 26140 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579: Neither blocks nor log segments found. Creating new log.
I20260808 06:07:35.815462 26140 log.cc:824] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579: Log is configured to *not* fsync() on all Append() calls
I20260808 06:07:35.817672 26140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579: No bootstrap required, opened a new log
I20260808 06:07:35.821801 26140 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d12e98c2324449b8fbf2a5e12396579" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39415 } }
I20260808 06:07:35.822017 26140 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:07:35.822072 26140 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d12e98c2324449b8fbf2a5e12396579, State: Initialized, Role: FOLLOWER
Aug 08 06:07:35 dist-test-slave-zj8q krb5kdc[25996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169255, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:07:35.822513 26140 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d12e98c2324449b8fbf2a5e12396579" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39415 } }
I20260808 06:07:35.822665 26140 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:07:35.822733 26140 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:07:35.822836 26140 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:07:35.823935 26140 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d12e98c2324449b8fbf2a5e12396579" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39415 } }
I20260808 06:07:35.824301 26140 leader_election.cc:302] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d12e98c2324449b8fbf2a5e12396579; no voters:
I20260808 06:07:35.824640 26140 leader_election.cc:288] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:07:35.824721 26145 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:07:35.824918 26145 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [term 1 LEADER]: Becoming Leader. State: Replica: 1d12e98c2324449b8fbf2a5e12396579, State: Running, Role: LEADER
I20260808 06:07:35.825268 26145 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d12e98c2324449b8fbf2a5e12396579" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39415 } }
I20260808 06:07:35.825968 26140 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:07:35.827006 26146 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d12e98c2324449b8fbf2a5e12396579" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d12e98c2324449b8fbf2a5e12396579" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39415 } } }
I20260808 06:07:35.827147 26146 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [sys.catalog]: This master's current role is: LEADER
I20260808 06:07:35.827540 26148 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d12e98c2324449b8fbf2a5e12396579. Latest consensus state: current_term: 1 leader_uuid: "1d12e98c2324449b8fbf2a5e12396579" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d12e98c2324449b8fbf2a5e12396579" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39415 } } }
I20260808 06:07:35.827696 26148 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [sys.catalog]: This master's current role is: LEADER
I20260808 06:07:35.828635 26143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:35.814239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60562 (local address 127.1.151.190:39415)
0808 06:07:35.814710 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:35.814731 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:35.814778 (+ 47us) server_negotiation.cc:407] Connection header received
0808 06:07:35.815587 (+ 809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:35.815611 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:35.815999 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:35.816328 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:35.816974 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:35.818453 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:35.819337 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:35.820227 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:35.820396 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:35.822842 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:35.822848 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:35.822864 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:35.822898 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:35.824875 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:35.825379 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:35.825385 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:35.825392 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:35.825552 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:35.825865 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:35.825869 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:35.825870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:35.826423 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:35.827245 (+ 822us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:35.827404 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 06:07:35.827725 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":134,"threads_started":1}
W20260808 06:07:35.831156 26170 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 06:07:35.831223 26170 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 06:07:35.831306 26162 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:07:35.832013 26162 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:07:35.837023 26162 catalog_manager.cc:1347] Generated new cluster ID: 2b628fe1aedf4d5bbd03ad57895d53a4
I20260808 06:07:35.837086 26162 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:07:35.842016 26162 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:07:35.842854 26162 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:07:35.852201 26162 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579: Generated new TSK 0
I20260808 06:07:35.852857 26162 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.151.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab.
Entry for principal kudu/127.1.151.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab.
I20260808 06:07:35.884984 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.129:0
--local_ip_for_outbound_sockets=127.1.151.129
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:39415
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:35001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:07:36.029770 26176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:07:36.030007 26176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:07:36.030050 26176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:07:36.035264 26176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:07:36.035348 26176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:07:36.035483 26176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.129
I20260808 06:07:36.043061 26176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:35001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:39415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.129
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:07:36.044710 26176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:07:36.046448 26176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:07:36.061512 26183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:36.061633 26186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:36.061512 26184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:36.062364 26176 server_base.cc:1034] running on GCE node
I20260808 06:07:36.062973 26176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:07:36.063962 26176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:07:36.065166 26176 hybrid_clock.cc:648] HybridClock initialized: now 1786169256065109 us; error 73 us; skew 500 ppm
Aug 08 06:07:36 dist-test-slave-zj8q krb5kdc[25996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169256, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:07:36.069561 26176 init.cc:377] Logged in from keytab as kudu/127.1.151.129@KRBTEST.COM (short username kudu)
I20260808 06:07:36.070994 26176 webserver.cc:460] Webserver started at http://127.1.151.129:46077/ using document root <none> and password file <none>
I20260808 06:07:36.071982 26176 fs_manager.cc:359] Metadata directory not provided
I20260808 06:07:36.072105 26176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:07:36.072490 26176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:07:36.075941 26176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/data/instance:
uuid: "d0e15846c2fc43a29763dee9c94558dd"
format_stamp: "Formatted at 2026-08-08 06:07:36 on dist-test-slave-zj8q"
server_key: "995db3cfdead3e101bee455148f471fd"
server_key_iv: "8e8e1500e7dc48b641ed5aa515a6449c"
server_key_version: "encryptionkey@0"
I20260808 06:07:36.076975 26176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance:
uuid: "d0e15846c2fc43a29763dee9c94558dd"
format_stamp: "Formatted at 2026-08-08 06:07:36 on dist-test-slave-zj8q"
server_key: "995db3cfdead3e101bee455148f471fd"
server_key_iv: "8e8e1500e7dc48b641ed5aa515a6449c"
server_key_version: "encryptionkey@0"
I20260808 06:07:36.083400 26176 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260808 06:07:36.087339 26199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:36.089284 26176 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260808 06:07:36.089439 26176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/wal
uuid: "d0e15846c2fc43a29763dee9c94558dd"
format_stamp: "Formatted at 2026-08-08 06:07:36 on dist-test-slave-zj8q"
server_key: "995db3cfdead3e101bee455148f471fd"
server_key_iv: "8e8e1500e7dc48b641ed5aa515a6449c"
server_key_version: "encryptionkey@0"
I20260808 06:07:36.089593 26176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:07:36.109591 26176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:07:36.114274 26176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:07:36.114549 26176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:07:36.115482 26176 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:07:36.116843 26176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:07:36.116930 26176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:36.117065 26176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:07:36.117127 26176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:36.132970 26176 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.129:33379
I20260808 06:07:36.133013 26397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.129:33379 every 8 connection(s)
I20260808 06:07:36.134428 26176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
I20260808 06:07:36.143072 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 26176
I20260808 06:07:36.143278 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance
I20260808 06:07:36.143698 1630 external_mini_cluster.cc:1428] Setting key b37799e5f487143a31c46f7b62de5bd7
Aug 08 06:07:36 dist-test-slave-zj8q krb5kdc[25996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169256, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:07:36.158056 26143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:36.137014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:37011 (local address 127.1.151.190:39415)
0808 06:07:36.137297 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:36.137310 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:36.138836 (+ 1526us) server_negotiation.cc:407] Connection header received
0808 06:07:36.140913 (+ 2077us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:36.140917 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:36.140996 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:36.141089 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:36.143332 (+ 2243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:36.144114 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:36.146694 (+ 2580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:36.147285 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:36.147424 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:36.151666 (+ 4242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:36.151672 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:36.151680 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:36.151721 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:36.154002 (+ 2281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:36.155248 (+ 1246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:36.155255 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:36.155256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:36.155370 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:36.156015 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:36.156017 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:36.156018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:36.156348 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:36.156458 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:36.157691 (+ 1233us) server_negotiation.cc:299] Negotiation successful
0808 06:07:36.157869 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 06:07:36.160533 26404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:36.137819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.129:37011)
0808 06:07:36.138601 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:36.138698 (+ 97us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:36.140592 (+ 1894us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:36.141394 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:36.141419 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:36.142050 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:07:36.143127 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:36.143164 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:36.144273 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:36.144276 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:36.146449 (+ 2173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:36.146469 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:36.147606 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:36.147612 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:36.147901 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:36.148692 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:07:36.148720 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:07:36.151346 (+ 2626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:07:36.154264 (+ 2918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:36.154275 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:36.154305 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:36.155105 (+ 800us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:36.155494 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:36.155499 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:36.155501 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:36.155877 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:36.156508 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:07:36.156518 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:07:36.157235 (+ 717us) client_negotiation.cc:769] Sending connection context
0808 06:07:36.157629 (+ 394us) client_negotiation.cc:240] Negotiation successful
0808 06:07:36.158377 (+ 748us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":217,"threads_started":1}
WARNING: no policy specified for kudu/127.1.151.130@KRBTEST.COM; defaulting to no policy
I20260808 06:07:36.162524 26401 heartbeater.cc:347] Connected to a master server at 127.1.151.190:39415
I20260808 06:07:36.162973 26401 heartbeater.cc:464] Registering TS with master...
I20260808 06:07:36.163782 26401 heartbeater.cc:511] Master 127.1.151.190:39415 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.130@KRBTEST.COM" created.
I20260808 06:07:36.166062 26083 ts_manager.cc:194] Registered new tserver with Master: d0e15846c2fc43a29763dee9c94558dd (127.1.151.129:33379)
I20260808 06:07:36.168350 26083 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.129@KRBTEST.COM'} at 127.1.151.129:37011
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab.
Entry for principal kudu/127.1.151.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab.
I20260808 06:07:36.185664 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.130:0
--local_ip_for_outbound_sockets=127.1.151.130
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:39415
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:35001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:07:36.356707 26408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:07:36.357089 26408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:07:36.357198 26408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:07:36.362872 26408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:07:36.362946 26408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:07:36.363056 26408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.130
I20260808 06:07:36.370440 26408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:35001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:39415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.130
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:07:36.372062 26408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:07:36.373596 26408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:07:36.388355 26422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:36.388363 26424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:36.388361 26426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:36.388813 26408 server_base.cc:1034] running on GCE node
I20260808 06:07:36.389523 26408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:07:36.390527 26408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:07:36.391753 26408 hybrid_clock.cc:648] HybridClock initialized: now 1786169256391711 us; error 48 us; skew 500 ppm
Aug 08 06:07:36 dist-test-slave-zj8q krb5kdc[25996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169256, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:07:36.396129 26408 init.cc:377] Logged in from keytab as kudu/127.1.151.130@KRBTEST.COM (short username kudu)
I20260808 06:07:36.397734 26408 webserver.cc:460] Webserver started at http://127.1.151.130:36177/ using document root <none> and password file <none>
I20260808 06:07:36.398743 26408 fs_manager.cc:359] Metadata directory not provided
I20260808 06:07:36.398861 26408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:07:36.399206 26408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:07:36.402560 26408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/data/instance:
uuid: "05cb149b8a664d0b89186253ff773f8b"
format_stamp: "Formatted at 2026-08-08 06:07:36 on dist-test-slave-zj8q"
server_key: "7e3281fcb12fe62372dd49fdb7451ea4"
server_key_iv: "4614d4e9c7bab6ef96db0e46cc08cd4c"
server_key_version: "encryptionkey@0"
I20260808 06:07:36.403415 26408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance:
uuid: "05cb149b8a664d0b89186253ff773f8b"
format_stamp: "Formatted at 2026-08-08 06:07:36 on dist-test-slave-zj8q"
server_key: "7e3281fcb12fe62372dd49fdb7451ea4"
server_key_iv: "4614d4e9c7bab6ef96db0e46cc08cd4c"
server_key_version: "encryptionkey@0"
I20260808 06:07:36.409647 26408 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260808 06:07:36.413705 26440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:36.415725 26408 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 06:07:36.415906 26408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/wal
uuid: "05cb149b8a664d0b89186253ff773f8b"
format_stamp: "Formatted at 2026-08-08 06:07:36 on dist-test-slave-zj8q"
server_key: "7e3281fcb12fe62372dd49fdb7451ea4"
server_key_iv: "4614d4e9c7bab6ef96db0e46cc08cd4c"
server_key_version: "encryptionkey@0"
I20260808 06:07:36.416052 26408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:07:36.432578 26408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:07:36.437000 26408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:07:36.437268 26408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:07:36.438236 26408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:07:36.439585 26408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:07:36.439672 26408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:36.439776 26408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:07:36.439838 26408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:36.454989 26408 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.130:45151
I20260808 06:07:36.455050 26644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.130:45151 every 8 connection(s)
I20260808 06:07:36.456450 26408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
I20260808 06:07:36.466291 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 26408
I20260808 06:07:36.466466 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance
I20260808 06:07:36.467057 1630 external_mini_cluster.cc:1428] Setting key 5418abd69b05cc0958f763d79d6f348e
Aug 08 06:07:36 dist-test-slave-zj8q krb5kdc[25996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169256, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:07:36.480417 26143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:36.459411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:56017 (local address 127.1.151.190:39415)
0808 06:07:36.459683 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:36.459708 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:36.461161 (+ 1453us) server_negotiation.cc:407] Connection header received
0808 06:07:36.463251 (+ 2090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:36.463254 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:36.463328 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:36.463444 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:36.465357 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:36.466305 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:36.468869 (+ 2564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:36.469379 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:36.469512 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:36.474377 (+ 4865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:36.474382 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:36.474389 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:36.474419 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:36.476887 (+ 2468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:36.477788 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:36.477793 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:36.477795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:36.477922 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:36.478490 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:36.478500 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:36.478502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:36.478822 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:36.478990 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:36.479849 (+ 859us) server_negotiation.cc:299] Negotiation successful
0808 06:07:36.480063 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 06:07:36.481806 26647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:36.460230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.130:56017)
0808 06:07:36.460944 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:36.461007 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:36.462921 (+ 1914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:36.463718 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:36.463737 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:36.464357 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:07:36.465155 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:36.465180 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:36.466490 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:36.466495 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:36.468681 (+ 2186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:36.468699 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:36.469622 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:36.469629 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:36.470083 (+ 454us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:36.471105 (+ 1022us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:07:36.471150 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:07:36.474097 (+ 2947us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:07:36.477081 (+ 2984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:36.477097 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:36.477113 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:36.477643 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:36.478048 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:36.478053 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:36.478055 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:36.478355 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:36.478987 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:07:36.478996 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:07:36.479419 (+ 423us) client_negotiation.cc:769] Sending connection context
0808 06:07:36.479713 (+ 294us) client_negotiation.cc:240] Negotiation successful
0808 06:07:36.480369 (+ 656us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":185,"threads_started":1}
WARNING: no policy specified for kudu/127.1.151.131@KRBTEST.COM; defaulting to no policy
I20260808 06:07:36.483697 26645 heartbeater.cc:347] Connected to a master server at 127.1.151.190:39415
I20260808 06:07:36.484059 26645 heartbeater.cc:464] Registering TS with master...
I20260808 06:07:36.484844 26645 heartbeater.cc:511] Master 127.1.151.190:39415 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.131@KRBTEST.COM" created.
I20260808 06:07:36.486508 26083 ts_manager.cc:194] Registered new tserver with Master: 05cb149b8a664d0b89186253ff773f8b (127.1.151.130:45151)
I20260808 06:07:36.487366 26083 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.130@KRBTEST.COM'} at 127.1.151.130:56017
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab.
Entry for principal kudu/127.1.151.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab.
I20260808 06:07:36.505507 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.131:0
--local_ip_for_outbound_sockets=127.1.151.131
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:39415
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:35001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:07:36.691411 26651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:07:36.691823 26651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:07:36.691905 26651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:07:36.698094 26651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:07:36.698230 26651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:07:36.698366 26651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.131
I20260808 06:07:36.706641 26651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:35001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:39415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.131
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:07:36.708369 26651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:07:36.710034 26651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:07:36.725781 26661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:36.725942 26663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:36.725836 26659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:36.726475 26651 server_base.cc:1034] running on GCE node
I20260808 06:07:36.727224 26651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:07:36.728312 26651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:07:36.729518 26651 hybrid_clock.cc:648] HybridClock initialized: now 1786169256729454 us; error 75 us; skew 500 ppm
Aug 08 06:07:36 dist-test-slave-zj8q krb5kdc[25996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169256, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:07:36.733368 26651 init.cc:377] Logged in from keytab as kudu/127.1.151.131@KRBTEST.COM (short username kudu)
I20260808 06:07:36.734688 26651 webserver.cc:460] Webserver started at http://127.1.151.131:38245/ using document root <none> and password file <none>
I20260808 06:07:36.735502 26651 fs_manager.cc:359] Metadata directory not provided
I20260808 06:07:36.735615 26651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:07:36.735945 26651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:07:36.739087 26651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/data/instance:
uuid: "e9e76571d8944ab7a9226db3a83a8719"
format_stamp: "Formatted at 2026-08-08 06:07:36 on dist-test-slave-zj8q"
server_key: "54b70e21e3349c4ac35822ad6ed91275"
server_key_iv: "fe06e4a73ea64348933d25d7624ccf39"
server_key_version: "encryptionkey@0"
I20260808 06:07:36.739974 26651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance:
uuid: "e9e76571d8944ab7a9226db3a83a8719"
format_stamp: "Formatted at 2026-08-08 06:07:36 on dist-test-slave-zj8q"
server_key: "54b70e21e3349c4ac35822ad6ed91275"
server_key_iv: "fe06e4a73ea64348933d25d7624ccf39"
server_key_version: "encryptionkey@0"
I20260808 06:07:36.745453 26651 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 06:07:36.748986 26677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:36.750864 26651 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 06:07:36.751029 26651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/wal
uuid: "e9e76571d8944ab7a9226db3a83a8719"
format_stamp: "Formatted at 2026-08-08 06:07:36 on dist-test-slave-zj8q"
server_key: "54b70e21e3349c4ac35822ad6ed91275"
server_key_iv: "fe06e4a73ea64348933d25d7624ccf39"
server_key_version: "encryptionkey@0"
I20260808 06:07:36.751185 26651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:07:36.783705 26651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:07:36.791031 26651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:07:36.793155 26651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:07:36.794127 26651 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:07:36.795825 26651 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:07:36.795910 26651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:36.796008 26651 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:07:36.796061 26651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:36.815706 26651 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.131:46043
I20260808 06:07:36.815770 26867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.131:46043 every 8 connection(s)
I20260808 06:07:36.817420 26651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
I20260808 06:07:36.830286 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 26651
I20260808 06:07:36.830483 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance
I20260808 06:07:36.831071 1630 external_mini_cluster.cc:1428] Setting key 7e9d240bc91eb660e972088744f3385f
Aug 08 06:07:36 dist-test-slave-zj8q krb5kdc[25996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169256, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:07:36.848532 26143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:36.820877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:54385 (local address 127.1.151.190:39415)
0808 06:07:36.821058 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:36.821063 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:36.823069 (+ 2006us) server_negotiation.cc:407] Connection header received
0808 06:07:36.826360 (+ 3291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:36.826364 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:36.826446 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:36.826620 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:36.830127 (+ 3507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:36.831155 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:36.834354 (+ 3199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:36.834923 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:36.835128 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:36.840507 (+ 5379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:36.840521 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:36.840526 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:36.840575 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:36.843664 (+ 3089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:36.844693 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:36.844698 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:36.844699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:36.844860 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:36.845668 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:36.845682 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:36.845693 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:36.846177 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:36.846298 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:36.848016 (+ 1718us) server_negotiation.cc:299] Negotiation successful
0808 06:07:36.848248 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 06:07:36.850339 26872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:36.821784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.131:54385)
0808 06:07:36.822756 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:36.822863 (+ 107us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:36.825861 (+ 2998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:36.826889 (+ 1028us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:36.826906 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:36.828131 (+ 1225us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:07:36.829845 (+ 1714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:36.829880 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:36.831423 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:36.831437 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:36.834054 (+ 2617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:36.834080 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:36.835576 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:36.835582 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:36.836002 (+ 420us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:36.837122 (+ 1120us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:07:36.837172 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:07:36.840216 (+ 3044us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:07:36.843869 (+ 3653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:36.843878 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:36.843909 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:36.844512 (+ 603us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:36.845097 (+ 585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:36.845100 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:36.845102 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:36.845451 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:36.846308 (+ 857us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:07:36.846319 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:07:36.847578 (+ 1259us) client_negotiation.cc:769] Sending connection context
0808 06:07:36.847870 (+ 292us) client_negotiation.cc:240] Negotiation successful
0808 06:07:36.848418 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":246,"threads_started":1}
I20260808 06:07:36.852867 26868 heartbeater.cc:347] Connected to a master server at 127.1.151.190:39415
I20260808 06:07:36.853336 26868 heartbeater.cc:464] Registering TS with master...
I20260808 06:07:36.854395 26868 heartbeater.cc:511] Master 127.1.151.190:39415 requested a full tablet report, sending...
I20260808 06:07:36.856268 26083 ts_manager.cc:194] Registered new tserver with Master: e9e76571d8944ab7a9226db3a83a8719 (127.1.151.131:46043)
I20260808 06:07:36.857493 26083 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.131@KRBTEST.COM'} at 127.1.151.131:54385
I20260808 06:07:36.860533 1630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 06:07:36.878072 26143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:36.863908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60568 (local address 127.1.151.190:39415)
0808 06:07:36.864119 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:36.864124 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:36.864483 (+ 359us) server_negotiation.cc:407] Connection header received
0808 06:07:36.864764 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:36.864767 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:36.864863 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:36.865000 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:36.866779 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:36.867875 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:36.869770 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:36.870271 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:36.870416 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:36.872646 (+ 2230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:36.872659 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:36.872662 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:36.872700 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:36.875174 (+ 2474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:36.875781 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:36.875793 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:36.875796 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:36.875943 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:36.876389 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:36.876395 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:36.876397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:36.876867 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:36.877069 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:36.877597 (+ 528us) server_negotiation.cc:299] Negotiation successful
0808 06:07:36.877785 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 06:07:36.883028 26083 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60568:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 06:07:36.889068 26083 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 06:07:36.912849 26883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:36.902653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:46043 (local address 127.0.0.1:33998)
0808 06:07:36.903780 (+ 1127us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:36.903804 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:36.903951 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:36.905463 (+ 1512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:36.905471 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:36.905512 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:07:36.905736 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:36.905761 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:36.908147 (+ 2386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:36.908150 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:36.910522 (+ 2372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:36.910534 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:36.912241 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:36.912247 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:36.912384 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:36.912428 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 06:07:36.912543 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 06:07:36.912672 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1022,"mutex_wait_us":327,"thread_start_us":178,"threads_started":1}
I20260808 06:07:36.913048 26889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:36.903753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33998 (local address 127.1.151.131:46043)
0808 06:07:36.904552 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:36.904578 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:36.904639 (+ 61us) server_negotiation.cc:407] Connection header received
0808 06:07:36.904783 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:36.904788 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:36.905159 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:36.905485 (+ 326us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:07:36.905976 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:36.907847 (+ 1871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:36.910721 (+ 2874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:36.911965 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:36.912231 (+ 266us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:36.912477 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:36.912623 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 06:07:36.912798 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":250,"threads_started":1}
I20260808 06:07:36.913244 26886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:36.902679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58830 (local address 127.1.151.130:45151)
0808 06:07:36.903404 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:36.903429 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:36.903506 (+ 77us) server_negotiation.cc:407] Connection header received
0808 06:07:36.903954 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:36.903973 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:36.904375 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:36.904673 (+ 298us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:07:36.906197 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:36.907906 (+ 1709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:36.910382 (+ 2476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:36.911737 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:36.911856 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:36.912166 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:36.912804 (+ 638us) server_negotiation.cc:299] Negotiation successful
0808 06:07:36.912968 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":195,"threads_started":1}
I20260808 06:07:36.913635 26892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:36.904647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51926 (local address 127.1.151.129:33379)
0808 06:07:36.905075 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:36.905085 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:36.905318 (+ 233us) server_negotiation.cc:407] Connection header received
0808 06:07:36.905653 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:36.905661 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:36.905995 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:36.906693 (+ 698us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:07:36.906867 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:36.907838 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:36.910771 (+ 2933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:36.911616 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:36.911769 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:36.912065 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:36.913080 (+ 1015us) server_negotiation.cc:299] Negotiation successful
0808 06:07:36.913266 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":104,"threads_started":1}
I20260808 06:07:36.913908 26885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:36.902428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:45151 (local address 127.0.0.1:58830)
0808 06:07:36.903298 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:36.903383 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:36.903598 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:36.905701 (+ 2103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:36.905705 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:36.905735 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:07:36.905899 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:36.905918 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:36.908177 (+ 2259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:36.908180 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:36.910128 (+ 1948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:36.910139 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:36.912327 (+ 2188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:36.912332 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:36.912509 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:36.912580 (+ 71us) client_negotiation.cc:769] Sending connection context
0808 06:07:36.913529 (+ 949us) client_negotiation.cc:240] Negotiation successful
0808 06:07:36.913664 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"mutex_wait_us":286,"thread_start_us":231,"threads_started":1}
I20260808 06:07:36.912750 26891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:36.904420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:33379 (local address 127.0.0.1:51926)
0808 06:07:36.905194 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:36.905215 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:36.905366 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:36.906244 (+ 878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:36.906247 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:36.906262 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:07:36.906413 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:36.906431 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:36.907995 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:36.908001 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:36.910530 (+ 2529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:36.910537 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:36.911765 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:36.911768 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:36.911942 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:36.912032 (+ 90us) client_negotiation.cc:769] Sending connection context
0808 06:07:36.912226 (+ 194us) client_negotiation.cc:240] Negotiation successful
0808 06:07:36.912441 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"mutex_wait_us":317,"thread_start_us":115,"threads_started":1}
I20260808 06:07:36.919618 26282 tablet_service.cc:1467] Processing CreateTablet for tablet 9d4b6dabf64a4324a7e4b837855a1bcb (DEFAULT_TABLE table=test-table [id=12900096973841ceaa8a9fbe525a8ba3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:07:36.919348 26751 tablet_service.cc:1467] Processing CreateTablet for tablet 9d4b6dabf64a4324a7e4b837855a1bcb (DEFAULT_TABLE table=test-table [id=12900096973841ceaa8a9fbe525a8ba3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:07:36.919356 26532 tablet_service.cc:1467] Processing CreateTablet for tablet 9d4b6dabf64a4324a7e4b837855a1bcb (DEFAULT_TABLE table=test-table [id=12900096973841ceaa8a9fbe525a8ba3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:07:36.921185 26282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d4b6dabf64a4324a7e4b837855a1bcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:07:36.921360 26751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d4b6dabf64a4324a7e4b837855a1bcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:07:36.921954 26532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d4b6dabf64a4324a7e4b837855a1bcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:07:36.931532 26893 tablet_bootstrap.cc:492] T 9d4b6dabf64a4324a7e4b837855a1bcb P d0e15846c2fc43a29763dee9c94558dd: Bootstrap starting.
I20260808 06:07:36.936106 26895 tablet_bootstrap.cc:492] T 9d4b6dabf64a4324a7e4b837855a1bcb P 05cb149b8a664d0b89186253ff773f8b: Bootstrap starting.
I20260808 06:07:36.936774 26893 tablet_bootstrap.cc:654] T 9d4b6dabf64a4324a7e4b837855a1bcb P d0e15846c2fc43a29763dee9c94558dd: Neither blocks nor log segments found. Creating new log.
I20260808 06:07:36.937885 26893 log.cc:824] T 9d4b6dabf64a4324a7e4b837855a1bcb P d0e15846c2fc43a29763dee9c94558dd: Log is configured to *not* fsync() on all Append() calls
I20260808 06:07:36.938716 26901 tablet_bootstrap.cc:492] T 9d4b6dabf64a4324a7e4b837855a1bcb P e9e76571d8944ab7a9226db3a83a8719: Bootstrap starting.
I20260808 06:07:36.939117 26895 tablet_bootstrap.cc:654] T 9d4b6dabf64a4324a7e4b837855a1bcb P 05cb149b8a664d0b89186253ff773f8b: Neither blocks nor log segments found. Creating new log.
I20260808 06:07:36.940572 26893 tablet_bootstrap.cc:492] T 9d4b6dabf64a4324a7e4b837855a1bcb P d0e15846c2fc43a29763dee9c94558dd: No bootstrap required, opened a new log
I20260808 06:07:36.940935 26893 ts_tablet_manager.cc:1397] T 9d4b6dabf64a4324a7e4b837855a1bcb P d0e15846c2fc43a29763dee9c94558dd: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260808 06:07:36.941340 26895 log.cc:824] T 9d4b6dabf64a4324a7e4b837855a1bcb P 05cb149b8a664d0b89186253ff773f8b: Log is configured to *not* fsync() on all Append() calls
I20260808 06:07:36.943288 26901 tablet_bootstrap.cc:654] T 9d4b6dabf64a4324a7e4b837855a1bcb P e9e76571d8944ab7a9226db3a83a8719: Neither blocks nor log segments found. Creating new log.
I20260808 06:07:36.944722 26901 log.cc:824] T 9d4b6dabf64a4324a7e4b837855a1bcb P e9e76571d8944ab7a9226db3a83a8719: Log is configured to *not* fsync() on all Append() calls
I20260808 06:07:36.944940 26895 tablet_bootstrap.cc:492] T 9d4b6dabf64a4324a7e4b837855a1bcb P 05cb149b8a664d0b89186253ff773f8b: No bootstrap required, opened a new log
I20260808 06:07:36.945263 26895 ts_tablet_manager.cc:1397] T 9d4b6dabf64a4324a7e4b837855a1bcb P 05cb149b8a664d0b89186253ff773f8b: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260808 06:07:36.947014 26893 raft_consensus.cc:348] T 9d4b6dabf64a4324a7e4b837855a1bcb P d0e15846c2fc43a29763dee9c94558dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05cb149b8a664d0b89186253ff773f8b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45151 } } peers { permanent_uuid: "e9e76571d8944ab7a9226db3a83a8719" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 46043 } } peers { permanent_uuid: "d0e15846c2fc43a29763dee9c94558dd" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 33379 } }
I20260808 06:07:36.947415 26893 raft_consensus.cc:374] T 9d4b6dabf64a4324a7e4b837855a1bcb P d0e15846c2fc43a29763dee9c94558dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:07:36.947563 26893 raft_consensus.cc:729] T 9d4b6dabf64a4324a7e4b837855a1bcb P d0e15846c2fc43a29763dee9c94558dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0e15846c2fc43a29763dee9c94558dd, State: Initialized, Role: FOLLOWER
I20260808 06:07:36.948297 26901 tablet_bootstrap.cc:492] T 9d4b6dabf64a4324a7e4b837855a1bcb P e9e76571d8944ab7a9226db3a83a8719: No bootstrap required, opened a new log
I20260808 06:07:36.948221 26893 consensus_queue.cc:260] T 9d4b6dabf64a4324a7e4b837855a1bcb P d0e15846c2fc43a29763dee9c94558dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05cb149b8a664d0b89186253ff773f8b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45151 } } peers { permanent_uuid: "e9e76571d8944ab7a9226db3a83a8719" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 46043 } } peers { permanent_uuid: "d0e15846c2fc43a29763dee9c94558dd" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 33379 } }
I20260808 06:07:36.948604 26901 ts_tablet_manager.cc:1397] T 9d4b6dabf64a4324a7e4b837855a1bcb P e9e76571d8944ab7a9226db3a83a8719: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260808 06:07:36.949260 26401 heartbeater.cc:503] Master 127.1.151.190:39415 was elected leader, sending a full tablet report...
I20260808 06:07:36.949661 26893 ts_tablet_manager.cc:1428] T 9d4b6dabf64a4324a7e4b837855a1bcb P d0e15846c2fc43a29763dee9c94558dd: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260808 06:07:36.952193 26895 raft_consensus.cc:348] T 9d4b6dabf64a4324a7e4b837855a1bcb P 05cb149b8a664d0b89186253ff773f8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05cb149b8a664d0b89186253ff773f8b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45151 } } peers { permanent_uuid: "e9e76571d8944ab7a9226db3a83a8719" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 46043 } } peers { permanent_uuid: "d0e15846c2fc43a29763dee9c94558dd" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 33379 } }
I20260808 06:07:36.952765 26895 raft_consensus.cc:374] T 9d4b6dabf64a4324a7e4b837855a1bcb P 05cb149b8a664d0b89186253ff773f8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:07:36.952874 26895 raft_consensus.cc:729] T 9d4b6dabf64a4324a7e4b837855a1bcb P 05cb149b8a664d0b89186253ff773f8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05cb149b8a664d0b89186253ff773f8b, State: Initialized, Role: FOLLOWER
I20260808 06:07:36.953568 26895 consensus_queue.cc:260] T 9d4b6dabf64a4324a7e4b837855a1bcb P 05cb149b8a664d0b89186253ff773f8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05cb149b8a664d0b89186253ff773f8b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45151 } } peers { permanent_uuid: "e9e76571d8944ab7a9226db3a83a8719" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 46043 } } peers { permanent_uuid: "d0e15846c2fc43a29763dee9c94558dd" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 33379 } }
I20260808 06:07:36.954578 26645 heartbeater.cc:503] Master 127.1.151.190:39415 was elected leader, sending a full tablet report...
I20260808 06:07:36.955163 26895 ts_tablet_manager.cc:1428] T 9d4b6dabf64a4324a7e4b837855a1bcb P 05cb149b8a664d0b89186253ff773f8b: Time spent starting tablet: real 0.010s user 0.008s sys 0.000s
I20260808 06:07:36.955327 26901 raft_consensus.cc:348] T 9d4b6dabf64a4324a7e4b837855a1bcb P e9e76571d8944ab7a9226db3a83a8719 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05cb149b8a664d0b89186253ff773f8b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45151 } } peers { permanent_uuid: "e9e76571d8944ab7a9226db3a83a8719" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 46043 } } peers { permanent_uuid: "d0e15846c2fc43a29763dee9c94558dd" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 33379 } }
I20260808 06:07:36.955833 26901 raft_consensus.cc:374] T 9d4b6dabf64a4324a7e4b837855a1bcb P e9e76571d8944ab7a9226db3a83a8719 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:07:36.955943 26901 raft_consensus.cc:729] T 9d4b6dabf64a4324a7e4b837855a1bcb P e9e76571d8944ab7a9226db3a83a8719 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9e76571d8944ab7a9226db3a83a8719, State: Initialized, Role: FOLLOWER
I20260808 06:07:36.956590 26901 consensus_queue.cc:260] T 9d4b6dabf64a4324a7e4b837855a1bcb P e9e76571d8944ab7a9226db3a83a8719 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05cb149b8a664d0b89186253ff773f8b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45151 } } peers { permanent_uuid: "e9e76571d8944ab7a9226db3a83a8719" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 46043 } } peers { permanent_uuid: "d0e15846c2fc43a29763dee9c94558dd" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 33379 } }
I20260808 06:07:36.957683 26868 heartbeater.cc:503] Master 127.1.151.190:39415 was elected leader, sending a full tablet report...
W20260808 06:07:36.958215 26646 tablet.cc:2369] T 9d4b6dabf64a4324a7e4b837855a1bcb P 05cb149b8a664d0b89186253ff773f8b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 06:07:36.958397 26901 ts_tablet_manager.cc:1428] T 9d4b6dabf64a4324a7e4b837855a1bcb P e9e76571d8944ab7a9226db3a83a8719: Time spent starting tablet: real 0.010s user 0.003s sys 0.006s
W20260808 06:07:37.069319 26870 tablet.cc:2369] T 9d4b6dabf64a4324a7e4b837855a1bcb P e9e76571d8944ab7a9226db3a83a8719: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 06:07:37.136577 26402 tablet.cc:2369] T 9d4b6dabf64a4324a7e4b837855a1bcb P d0e15846c2fc43a29763dee9c94558dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 06:07:37.145320 26912 raft_consensus.cc:482] T 9d4b6dabf64a4324a7e4b837855a1bcb P e9e76571d8944ab7a9226db3a83a8719 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:07:37.145632 26912 raft_consensus.cc:504] T 9d4b6dabf64a4324a7e4b837855a1bcb P e9e76571d8944ab7a9226db3a83a8719 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05cb149b8a664d0b89186253ff773f8b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45151 } } peers { permanent_uuid: "e9e76571d8944ab7a9226db3a83a8719" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 46043 } } peers { permanent_uuid: "d0e15846c2fc43a29763dee9c94558dd" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 33379 } }
I20260808 06:07:37.147334 26912 leader_election.cc:288] T 9d4b6dabf64a4324a7e4b837855a1bcb P e9e76571d8944ab7a9226db3a83a8719 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05cb149b8a664d0b89186253ff773f8b (127.1.151.130:45151), d0e15846c2fc43a29763dee9c94558dd (127.1.151.129:33379)
I20260808 06:07:37.154875 26892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:37.147752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:53847 (local address 127.1.151.129:33379)
0808 06:07:37.147949 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:37.147956 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:37.148431 (+ 475us) server_negotiation.cc:407] Connection header received
0808 06:07:37.148624 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:37.148628 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:37.148694 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:37.148773 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:07:37.149161 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:37.150319 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:37.152591 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:37.153636 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:37.153919 (+ 283us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:37.154029 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:37.154561 (+ 532us) server_negotiation.cc:299] Negotiation successful
0808 06:07:37.154679 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 06:07:37.154875 26886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:37.147834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:46529 (local address 127.1.151.130:45151)
0808 06:07:37.148045 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:37.148062 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:37.148088 (+ 26us) server_negotiation.cc:407] Connection header received
0808 06:07:37.148190 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:37.148202 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:37.148302 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:37.148446 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:07:37.148905 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:37.150090 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:37.152592 (+ 2502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:37.153653 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:37.153920 (+ 267us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:37.154028 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:37.154561 (+ 533us) server_negotiation.cc:299] Negotiation successful
0808 06:07:37.154680 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260808 06:07:37.154929 26872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:37.147609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:45151 (local address 127.1.151.131:46529)
0808 06:07:37.147795 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:37.147830 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:37.147966 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:37.148461 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:37.148474 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:37.148500 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:07:37.148720 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:37.148727 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:37.150297 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:37.150300 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:37.152405 (+ 2105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:37.152427 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:37.153896 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:37.153902 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:37.154336 (+ 434us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:37.154407 (+ 71us) client_negotiation.cc:769] Sending connection context
0808 06:07:37.154547 (+ 140us) client_negotiation.cc:240] Negotiation successful
0808 06:07:37.154690 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":69}
I20260808 06:07:37.154949 26913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:37.147624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:33379 (local address 127.1.151.131:53847)
0808 06:07:37.148305 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:37.148324 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:37.148461 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:37.148806 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:37.148809 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:37.148850 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:07:37.148999 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:37.149012 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:37.150441 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:37.150445 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:37.152423 (+ 1978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:37.152430 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:37.153896 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:37.153902 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:37.154354 (+ 452us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:37.154420 (+ 66us) client_negotiation.cc:769] Sending connection context
0808 06:07:37.154547 (+ 127us) client_negotiation.cc:240] Negotiation successful
0808 06:07:37.154690 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"mutex_wait_us":457,"thread_start_us":216,"threads_started":1}
I20260808 06:07:37.156286 26313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d4b6dabf64a4324a7e4b837855a1bcb" candidate_uuid: "e9e76571d8944ab7a9226db3a83a8719" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e15846c2fc43a29763dee9c94558dd" is_pre_election: true
I20260808 06:07:37.156293 26567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d4b6dabf64a4324a7e4b837855a1bcb" candidate_uuid: "e9e76571d8944ab7a9226db3a83a8719" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05cb149b8a664d0b89186253ff773f8b" is_pre_election: true
I20260808 06:07:37.156713 26567 raft_consensus.cc:2454] T 9d4b6dabf64a4324a7e4b837855a1bcb P 05cb149b8a664d0b89186253ff773f8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9e76571d8944ab7a9226db3a83a8719 in term 0.
I20260808 06:07:37.156715 26313 raft_consensus.cc:2454] T 9d4b6dabf64a4324a7e4b837855a1bcb P d0e15846c2fc43a29763dee9c94558dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9e76571d8944ab7a9226db3a83a8719 in term 0.
I20260808 06:07:37.157374 26681 leader_election.cc:302] T 9d4b6dabf64a4324a7e4b837855a1bcb P e9e76571d8944ab7a9226db3a83a8719 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0e15846c2fc43a29763dee9c94558dd, e9e76571d8944ab7a9226db3a83a8719; no voters:
I20260808 06:07:37.157795 26912 raft_consensus.cc:2781] T 9d4b6dabf64a4324a7e4b837855a1bcb P e9e76571d8944ab7a9226db3a83a8719 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 06:07:37.157964 26912 raft_consensus.cc:482] T 9d4b6dabf64a4324a7e4b837855a1bcb P e9e76571d8944ab7a9226db3a83a8719 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 06:07:37.158130 26912 raft_consensus.cc:3037] T 9d4b6dabf64a4324a7e4b837855a1bcb P e9e76571d8944ab7a9226db3a83a8719 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:07:37.159911 26912 raft_consensus.cc:504] T 9d4b6dabf64a4324a7e4b837855a1bcb P e9e76571d8944ab7a9226db3a83a8719 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05cb149b8a664d0b89186253ff773f8b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45151 } } peers { permanent_uuid: "e9e76571d8944ab7a9226db3a83a8719" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 46043 } } peers { permanent_uuid: "d0e15846c2fc43a29763dee9c94558dd" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 33379 } }
I20260808 06:07:37.160446 26912 leader_election.cc:288] T 9d4b6dabf64a4324a7e4b837855a1bcb P e9e76571d8944ab7a9226db3a83a8719 [CANDIDATE]: Term 1 election: Requested vote from peers 05cb149b8a664d0b89186253ff773f8b (127.1.151.130:45151), d0e15846c2fc43a29763dee9c94558dd (127.1.151.129:33379)
I20260808 06:07:37.160873 26567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d4b6dabf64a4324a7e4b837855a1bcb" candidate_uuid: "e9e76571d8944ab7a9226db3a83a8719" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05cb149b8a664d0b89186253ff773f8b"
I20260808 06:07:37.160920 26313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d4b6dabf64a4324a7e4b837855a1bcb" candidate_uuid: "e9e76571d8944ab7a9226db3a83a8719" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e15846c2fc43a29763dee9c94558dd"
I20260808 06:07:37.161103 26567 raft_consensus.cc:3037] T 9d4b6dabf64a4324a7e4b837855a1bcb P 05cb149b8a664d0b89186253ff773f8b [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:07:37.161123 26313 raft_consensus.cc:3037] T 9d4b6dabf64a4324a7e4b837855a1bcb P d0e15846c2fc43a29763dee9c94558dd [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:07:37.162653 26313 raft_consensus.cc:2454] T 9d4b6dabf64a4324a7e4b837855a1bcb P d0e15846c2fc43a29763dee9c94558dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9e76571d8944ab7a9226db3a83a8719 in term 1.
I20260808 06:07:37.162744 26567 raft_consensus.cc:2454] T 9d4b6dabf64a4324a7e4b837855a1bcb P 05cb149b8a664d0b89186253ff773f8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9e76571d8944ab7a9226db3a83a8719 in term 1.
I20260808 06:07:37.163285 26684 leader_election.cc:302] T 9d4b6dabf64a4324a7e4b837855a1bcb P e9e76571d8944ab7a9226db3a83a8719 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05cb149b8a664d0b89186253ff773f8b, e9e76571d8944ab7a9226db3a83a8719; no voters:
I20260808 06:07:37.163555 26912 raft_consensus.cc:2781] T 9d4b6dabf64a4324a7e4b837855a1bcb P e9e76571d8944ab7a9226db3a83a8719 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:07:37.163968 26912 raft_consensus.cc:686] T 9d4b6dabf64a4324a7e4b837855a1bcb P e9e76571d8944ab7a9226db3a83a8719 [term 1 LEADER]: Becoming Leader. State: Replica: e9e76571d8944ab7a9226db3a83a8719, State: Running, Role: LEADER
I20260808 06:07:37.164419 26912 consensus_queue.cc:237] T 9d4b6dabf64a4324a7e4b837855a1bcb P e9e76571d8944ab7a9226db3a83a8719 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05cb149b8a664d0b89186253ff773f8b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45151 } } peers { permanent_uuid: "e9e76571d8944ab7a9226db3a83a8719" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 46043 } } peers { permanent_uuid: "d0e15846c2fc43a29763dee9c94558dd" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 33379 } }
I20260808 06:07:37.169162 26078 catalog_manager.cc:5495] T 9d4b6dabf64a4324a7e4b837855a1bcb P e9e76571d8944ab7a9226db3a83a8719 reported cstate change: term changed from 0 to 1, leader changed from <none> to e9e76571d8944ab7a9226db3a83a8719 (127.1.151.131). New cstate: current_term: 1 leader_uuid: "e9e76571d8944ab7a9226db3a83a8719" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05cb149b8a664d0b89186253ff773f8b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9e76571d8944ab7a9226db3a83a8719" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 46043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0e15846c2fc43a29763dee9c94558dd" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 33379 } health_report { overall_health: UNKNOWN } } }
I20260808 06:07:37.209934 26889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:37.203613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34004 (local address 127.1.151.131:46043)
0808 06:07:37.203856 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:37.203869 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:37.203896 (+ 27us) server_negotiation.cc:407] Connection header received
0808 06:07:37.204096 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:37.204099 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:37.204218 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:37.204416 (+ 198us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:07:37.204831 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:37.205653 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:37.207001 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:37.207662 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:37.207836 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:37.208394 (+ 558us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:07:37.209282 (+ 888us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:07:37.209529 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:37.209611 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 06:07:37.209721 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260808 06:07:37.219678 26567 raft_consensus.cc:1261] T 9d4b6dabf64a4324a7e4b837855a1bcb P 05cb149b8a664d0b89186253ff773f8b [term 1 FOLLOWER]: Refusing update from remote peer e9e76571d8944ab7a9226db3a83a8719: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:07:37.219678 26313 raft_consensus.cc:1261] T 9d4b6dabf64a4324a7e4b837855a1bcb P d0e15846c2fc43a29763dee9c94558dd [term 1 FOLLOWER]: Refusing update from remote peer e9e76571d8944ab7a9226db3a83a8719: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:07:37.220750 26914 consensus_queue.cc:1035] T 9d4b6dabf64a4324a7e4b837855a1bcb P e9e76571d8944ab7a9226db3a83a8719 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0e15846c2fc43a29763dee9c94558dd" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 33379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:07:37.221011 26912 consensus_queue.cc:1035] T 9d4b6dabf64a4324a7e4b837855a1bcb P e9e76571d8944ab7a9226db3a83a8719 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05cb149b8a664d0b89186253ff773f8b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:07:37.235517 26922 mvcc.cc:205] Tried to move back new op lower bound from 7316149277560524800 to 7316149277349167104. Current Snapshot: MvccSnapshot[applied={T|T < 7316149277560524800}]
I20260808 06:07:37.238485 26923 mvcc.cc:205] Tried to move back new op lower bound from 7316149277560524800 to 7316149277349167104. Current Snapshot: MvccSnapshot[applied={T|T < 7316149277560524800}]
I20260808 06:07:37.248483 26083 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60568:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:07:37.248732 26083 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60568:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:07:37.252866 26083 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579: Sending DeleteTablet for 3 replicas of tablet 9d4b6dabf64a4324a7e4b837855a1bcb
I20260808 06:07:37.254298 26532 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d4b6dabf64a4324a7e4b837855a1bcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:07:37 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:58830
I20260808 06:07:37.254335 26751 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d4b6dabf64a4324a7e4b837855a1bcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:07:37 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:33998
I20260808 06:07:37.254391 26282 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d4b6dabf64a4324a7e4b837855a1bcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:07:37 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:51926
I20260808 06:07:37.255115 26932 tablet_replica.cc:330] T 9d4b6dabf64a4324a7e4b837855a1bcb P e9e76571d8944ab7a9226db3a83a8719: stopping tablet replica
I20260808 06:07:37.255458 26934 tablet_replica.cc:330] T 9d4b6dabf64a4324a7e4b837855a1bcb P 05cb149b8a664d0b89186253ff773f8b: stopping tablet replica
I20260808 06:07:37.255474 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 26176
I20260808 06:07:37.276042 26932 raft_consensus.cc:2229] T 9d4b6dabf64a4324a7e4b837855a1bcb P e9e76571d8944ab7a9226db3a83a8719 [term 1 LEADER]: Raft consensus shutting down.
I20260808 06:07:37.276983 26932 raft_consensus.cc:2258] T 9d4b6dabf64a4324a7e4b837855a1bcb P e9e76571d8944ab7a9226db3a83a8719 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:07:37.281423 26932 ts_tablet_manager.cc:1905] T 9d4b6dabf64a4324a7e4b837855a1bcb P e9e76571d8944ab7a9226db3a83a8719: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:07:37.286813 26934 raft_consensus.cc:2229] T 9d4b6dabf64a4324a7e4b837855a1bcb P 05cb149b8a664d0b89186253ff773f8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:07:37.286971 26932 ts_tablet_manager.cc:1918] T 9d4b6dabf64a4324a7e4b837855a1bcb P e9e76571d8944ab7a9226db3a83a8719: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:07:37.287145 26932 log.cc:1196] T 9d4b6dabf64a4324a7e4b837855a1bcb P e9e76571d8944ab7a9226db3a83a8719: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/wal/wals/9d4b6dabf64a4324a7e4b837855a1bcb
I20260808 06:07:37.287468 26934 raft_consensus.cc:2258] T 9d4b6dabf64a4324a7e4b837855a1bcb P 05cb149b8a664d0b89186253ff773f8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:07:37.287724 26932 ts_tablet_manager.cc:1939] T 9d4b6dabf64a4324a7e4b837855a1bcb P e9e76571d8944ab7a9226db3a83a8719: Deleting consensus metadata
I20260808 06:07:37.290020 26934 ts_tablet_manager.cc:1905] T 9d4b6dabf64a4324a7e4b837855a1bcb P 05cb149b8a664d0b89186253ff773f8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:07:37.290616 26054 catalog_manager.cc:4841] TS e9e76571d8944ab7a9226db3a83a8719 (127.1.151.131:46043): tablet 9d4b6dabf64a4324a7e4b837855a1bcb (table test-table [id=12900096973841ceaa8a9fbe525a8ba3]) successfully deleted
I20260808 06:07:37.294073 26934 ts_tablet_manager.cc:1918] T 9d4b6dabf64a4324a7e4b837855a1bcb P 05cb149b8a664d0b89186253ff773f8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:07:37.294299 26934 log.cc:1196] T 9d4b6dabf64a4324a7e4b837855a1bcb P 05cb149b8a664d0b89186253ff773f8b: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/wal/wals/9d4b6dabf64a4324a7e4b837855a1bcb
I20260808 06:07:37.294848 26934 ts_tablet_manager.cc:1939] T 9d4b6dabf64a4324a7e4b837855a1bcb P 05cb149b8a664d0b89186253ff773f8b: Deleting consensus metadata
I20260808 06:07:37.296617 26059 catalog_manager.cc:4841] TS 05cb149b8a664d0b89186253ff773f8b (127.1.151.130:45151): tablet 9d4b6dabf64a4324a7e4b837855a1bcb (table test-table [id=12900096973841ceaa8a9fbe525a8ba3]) successfully deleted
I20260808 06:07:37.298707 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 26408
W20260808 06:07:37.298708 26054 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.151.129:33379 (error 108)
I20260808 06:07:37.303354 26883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:37.302876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:33379 (local address 127.0.0.1:51928)
0808 06:07:37.303080 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:37.303197 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.129:33379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260808 06:07:37.303969 26054 catalog_manager.cc:4584] TS d0e15846c2fc43a29763dee9c94558dd (127.1.151.129:33379): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9d4b6dabf64a4324a7e4b837855a1bcb: Network error: Client connection negotiation failed: client connection to 127.1.151.129:33379: connect: Connection refused (error 111)
I20260808 06:07:37.316359 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 26651
I20260808 06:07:37.342097 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 26021
I20260808 06:07:57.360776 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:39415
--webserver_interface=127.1.151.190
--webserver_port=41189
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:35001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:39415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:07:57.476996 26937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:07:57.477247 26937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:07:57.477285 26937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:07:57.481130 26937 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 06:07:57.481172 26937 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 06:07:57.481187 26937 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 06:07:57.481199 26937 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:07:57.481215 26937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:07:57.481231 26937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:07:57.481248 26937 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:07:57.481276 26937 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:07:57.486459 26937 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:35001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:39415
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:39415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=41189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:07:57.487448 26937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:07:57.488430 26937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:07:57.502418 26948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:57.502441 26952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:57.502451 26949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:57.502731 26937 server_base.cc:1034] running on GCE node
I20260808 06:07:57.503330 26937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:07:57.504397 26937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:07:57.505570 26937 hybrid_clock.cc:648] HybridClock initialized: now 1786169277505556 us; error 32 us; skew 500 ppm
Aug 08 06:07:57 dist-test-slave-zj8q krb5kdc[25996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169277, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:07:57.508540 26937 init.cc:377] Logged in from keytab as kudu/127.1.151.190@KRBTEST.COM (short username kudu)
I20260808 06:07:57.509411 26937 webserver.cc:460] Webserver started at http://127.1.151.190:41189/ using document root <none> and password file <none>
I20260808 06:07:57.509950 26937 fs_manager.cc:359] Metadata directory not provided
I20260808 06:07:57.510022 26937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:07:57.513746 26937 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:07:57.515838 26968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:57.516993 26937 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 06:07:57.517122 26937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "1d12e98c2324449b8fbf2a5e12396579"
format_stamp: "Formatted at 2026-08-08 06:07:35 on dist-test-slave-zj8q"
server_key: "c15b5a4cf93da8907c0fddcdc9eaa2ca"
server_key_iv: "8f3e1e1f4fe612d95af6b4a87993804d"
server_key_version: "encryptionkey@0"
I20260808 06:07:57.517702 26937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:07:57.533939 26937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:07:57.536903 26937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:07:57.537078 26937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:07:57.544754 26937 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:39415
I20260808 06:07:57.545023 27057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:39415 every 8 connection(s)
I20260808 06:07:57.545910 26937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:07:57.549105 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 26937
I20260808 06:07:57.549605 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.129:33379
--local_ip_for_outbound_sockets=127.1.151.129
--tserver_master_addrs=127.1.151.190:39415
--webserver_port=46077
--webserver_interface=127.1.151.129
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:35001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 06:07:57.550748 27058 sys_catalog.cc:263] Verifying existing consensus state
I20260808 06:07:57.552587 27058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579: Bootstrap starting.
I20260808 06:07:57.571758 27058 log.cc:824] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579: Log is configured to *not* fsync() on all Append() calls
I20260808 06:07:57.581277 27058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 06:07:57.581660 27058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579: Bootstrap complete.
I20260808 06:07:57.585713 27058 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d12e98c2324449b8fbf2a5e12396579" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39415 } }
I20260808 06:07:57.586115 27058 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d12e98c2324449b8fbf2a5e12396579, State: Initialized, Role: FOLLOWER
I20260808 06:07:57.586601 27058 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d12e98c2324449b8fbf2a5e12396579" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39415 } }
I20260808 06:07:57.586724 27058 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:07:57.586794 27058 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:07:57.586884 27058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [term 1 FOLLOWER]: Advancing to term 2
I20260808 06:07:57.588725 27058 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d12e98c2324449b8fbf2a5e12396579" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39415 } }
I20260808 06:07:57.589032 27058 leader_election.cc:302] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d12e98c2324449b8fbf2a5e12396579; no voters:
I20260808 06:07:57.589315 27058 leader_election.cc:288] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 06:07:57.589411 27064 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 06:07:57.589653 27064 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [term 2 LEADER]: Becoming Leader. State: Replica: 1d12e98c2324449b8fbf2a5e12396579, State: Running, Role: LEADER
I20260808 06:07:57.589986 27064 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d12e98c2324449b8fbf2a5e12396579" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39415 } }
I20260808 06:07:57.590617 27058 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:07:57.590955 27064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d12e98c2324449b8fbf2a5e12396579. Latest consensus state: current_term: 2 leader_uuid: "1d12e98c2324449b8fbf2a5e12396579" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d12e98c2324449b8fbf2a5e12396579" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39415 } } }
I20260808 06:07:57.591084 27064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [sys.catalog]: This master's current role is: LEADER
I20260808 06:07:57.591840 27069 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:07:57.592706 27065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1d12e98c2324449b8fbf2a5e12396579" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d12e98c2324449b8fbf2a5e12396579" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39415 } } }
I20260808 06:07:57.592804 27065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [sys.catalog]: This master's current role is: LEADER
I20260808 06:07:57.596589 27069 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:07:57.597210 27069 catalog_manager.cc:1259] Loaded cluster ID: 2b628fe1aedf4d5bbd03ad57895d53a4
I20260808 06:07:57.597261 27069 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:07:57.599035 27069 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:07:57.599627 27069 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579: Loaded TSK: 0
I20260808 06:07:57.618283 27069 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579: Generated new TSK 1
I20260808 06:07:57.618530 27069 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 06:07:57.678642 27061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:07:57.678885 27061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:07:57.678942 27061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:07:57.682955 27061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:07:57.683007 27061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:07:57.683086 27061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.129
I20260808 06:07:57.688144 27061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:35001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.129:33379
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.151.129
--webserver_port=46077
--tserver_master_addrs=127.1.151.190:39415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.129
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:07:57.689129 27061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:07:57.690088 27061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:07:57.700860 27094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:57.700984 27061 server_base.cc:1034] running on GCE node
W20260808 06:07:57.700834 27093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:57.700846 27099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:57.701593 27061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:07:57.702256 27061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:07:57.703423 27061 hybrid_clock.cc:648] HybridClock initialized: now 1786169277703388 us; error 52 us; skew 500 ppm
Aug 08 06:07:57 dist-test-slave-zj8q krb5kdc[25996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169277, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:07:57.706058 27061 init.cc:377] Logged in from keytab as kudu/127.1.151.129@KRBTEST.COM (short username kudu)
I20260808 06:07:57.706905 27061 webserver.cc:460] Webserver started at http://127.1.151.129:46077/ using document root <none> and password file <none>
I20260808 06:07:57.707432 27061 fs_manager.cc:359] Metadata directory not provided
I20260808 06:07:57.707506 27061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:07:57.711341 27061 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260808 06:07:57.713420 27108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:57.714915 27061 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 06:07:57.715055 27061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/wal
uuid: "d0e15846c2fc43a29763dee9c94558dd"
format_stamp: "Formatted at 2026-08-08 06:07:36 on dist-test-slave-zj8q"
server_key: "995db3cfdead3e101bee455148f471fd"
server_key_iv: "8e8e1500e7dc48b641ed5aa515a6449c"
server_key_version: "encryptionkey@0"
I20260808 06:07:57.715761 27061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:07:57.755295 27061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:07:57.758855 27061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:07:57.759060 27061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:07:57.759799 27061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:07:57.760912 27123 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 06:07:57.765059 27061 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 06:07:57.765117 27061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260808 06:07:57.765174 27061 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 06:07:57.766830 27061 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 06:07:57.766876 27061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260808 06:07:57.767174 27123 tablet_bootstrap.cc:492] T 9d4b6dabf64a4324a7e4b837855a1bcb P d0e15846c2fc43a29763dee9c94558dd: Bootstrap starting.
I20260808 06:07:57.776124 27061 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.129:33379
I20260808 06:07:57.776263 27294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.129:33379 every 8 connection(s)
I20260808 06:07:57.777037 27061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
I20260808 06:07:57.784828 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 27061
I20260808 06:07:57.785240 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.130:45151
--local_ip_for_outbound_sockets=127.1.151.130
--tserver_master_addrs=127.1.151.190:39415
--webserver_port=36177
--webserver_interface=127.1.151.130
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:35001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 06:07:57.787163 27123 log.cc:824] T 9d4b6dabf64a4324a7e4b837855a1bcb P d0e15846c2fc43a29763dee9c94558dd: Log is configured to *not* fsync() on all Append() calls
Aug 08 06:07:57 dist-test-slave-zj8q krb5kdc[25996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169277, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:07:57.795252 27123 tablet_bootstrap.cc:492] T 9d4b6dabf64a4324a7e4b837855a1bcb P d0e15846c2fc43a29763dee9c94558dd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 06:07:57.795457 27300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:57.779224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:47393 (local address 127.1.151.190:39415)
0808 06:07:57.779680 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:57.779704 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:57.779792 (+ 88us) server_negotiation.cc:407] Connection header received
0808 06:07:57.781112 (+ 1320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:57.781122 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:57.781399 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:57.781662 (+ 263us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:57.783221 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:57.784200 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:57.785992 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:57.786681 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:57.786843 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:57.789589 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:57.789595 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:57.789611 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:57.789643 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:57.791827 (+ 2184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:57.792502 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:57.792509 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:57.792514 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:57.792672 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:57.793054 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:57.793057 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:57.793059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:57.793567 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:57.793703 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:57.794322 (+ 619us) server_negotiation.cc:299] Negotiation successful
0808 06:07:57.794621 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":99,"threads_started":1}
I20260808 06:07:57.795475 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:57.779065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.129:47393)
0808 06:07:57.779637 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:57.779693 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:57.780688 (+ 995us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:57.781796 (+ 1108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:57.781806 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:57.782415 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:07:57.783030 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:57.783049 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:57.784358 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:57.784361 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:57.785850 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:57.785863 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:57.786838 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:57.786842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:57.787033 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:57.787605 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:07:57.787625 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:07:57.789377 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:07:57.791952 (+ 2575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:57.791957 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:57.791968 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:57.792354 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:57.792775 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:57.792778 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:57.792780 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:57.792953 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:57.793700 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:07:57.793706 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:07:57.793971 (+ 265us) client_negotiation.cc:769] Sending connection context
0808 06:07:57.794212 (+ 241us) client_negotiation.cc:240] Negotiation successful
0808 06:07:57.794517 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":149,"threads_started":1}
I20260808 06:07:57.795732 27123 tablet_bootstrap.cc:492] T 9d4b6dabf64a4324a7e4b837855a1bcb P d0e15846c2fc43a29763dee9c94558dd: Bootstrap complete.
I20260808 06:07:57.796757 27123 ts_tablet_manager.cc:1397] T 9d4b6dabf64a4324a7e4b837855a1bcb P d0e15846c2fc43a29763dee9c94558dd: Time spent bootstrapping tablet: real 0.030s user 0.016s sys 0.012s
I20260808 06:07:57.798204 27295 heartbeater.cc:347] Connected to a master server at 127.1.151.190:39415
I20260808 06:07:57.798448 27295 heartbeater.cc:464] Registering TS with master...
I20260808 06:07:57.798978 27295 heartbeater.cc:511] Master 127.1.151.190:39415 requested a full tablet report, sending...
I20260808 06:07:57.799299 27123 raft_consensus.cc:348] T 9d4b6dabf64a4324a7e4b837855a1bcb P d0e15846c2fc43a29763dee9c94558dd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05cb149b8a664d0b89186253ff773f8b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45151 } } peers { permanent_uuid: "e9e76571d8944ab7a9226db3a83a8719" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 46043 } } peers { permanent_uuid: "d0e15846c2fc43a29763dee9c94558dd" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 33379 } }
I20260808 06:07:57.799701 27123 raft_consensus.cc:729] T 9d4b6dabf64a4324a7e4b837855a1bcb P d0e15846c2fc43a29763dee9c94558dd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0e15846c2fc43a29763dee9c94558dd, State: Initialized, Role: FOLLOWER
I20260808 06:07:57.800145 27123 consensus_queue.cc:260] T 9d4b6dabf64a4324a7e4b837855a1bcb P d0e15846c2fc43a29763dee9c94558dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05cb149b8a664d0b89186253ff773f8b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45151 } } peers { permanent_uuid: "e9e76571d8944ab7a9226db3a83a8719" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 46043 } } peers { permanent_uuid: "d0e15846c2fc43a29763dee9c94558dd" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 33379 } }
I20260808 06:07:57.801083 27123 ts_tablet_manager.cc:1428] T 9d4b6dabf64a4324a7e4b837855a1bcb P d0e15846c2fc43a29763dee9c94558dd: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260808 06:07:57.801628 26994 ts_manager.cc:194] Registered new tserver with Master: d0e15846c2fc43a29763dee9c94558dd (127.1.151.129:33379)
I20260808 06:07:57.805761 26994 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.129@KRBTEST.COM'} at 127.1.151.129:47393
I20260808 06:07:57.808382 27295 heartbeater.cc:503] Master 127.1.151.190:39415 was elected leader, sending a full tablet report...
Aug 08 06:07:57 dist-test-slave-zj8q krb5kdc[25996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169277, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for kudu/127.1.151.129@KRBTEST.COM
I20260808 06:07:57.814075 27307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:57.804722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:33379 (local address 127.0.0.1:55392)
0808 06:07:57.805075 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:57.805116 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:57.805245 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:57.805819 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:57.805825 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:57.806187 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:07:57.806299 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:57.806306 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:57.807341 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:57.807344 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:57.807947 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:57.807955 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:57.808496 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:57.808500 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:57.808606 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:57.809115 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:07:57.809129 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:07:57.810821 (+ 1692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:07:57.812470 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:57.812479 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:57.812485 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:57.812707 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:57.812986 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:57.812989 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:57.812991 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:57.813070 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:57.813487 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:07:57.813493 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:07:57.813603 (+ 110us) client_negotiation.cc:769] Sending connection context
0808 06:07:57.813721 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 06:07:57.813905 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":91,"threads_started":1}
I20260808 06:07:57.814103 27309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:57.804780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55392 (local address 127.1.151.129:33379)
0808 06:07:57.805426 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:57.805435 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:57.805452 (+ 17us) server_negotiation.cc:407] Connection header received
0808 06:07:57.805501 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:57.805505 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:57.805676 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:57.805793 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:57.806412 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:57.807234 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:57.808061 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:57.808388 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:57.808482 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:57.810953 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:57.810964 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:57.810976 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:57.811001 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:57.812367 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:57.812815 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:57.812821 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:57.812823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:57.812900 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:57.813151 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:57.813154 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:57.813156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:57.813376 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:57.813495 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:57.813819 (+ 324us) server_negotiation.cc:299] Negotiation successful
0808 06:07:57.813967 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":108,"threads_started":1}
I20260808 06:07:57.815322 27194 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d4b6dabf64a4324a7e4b837855a1bcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:07:37 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:55392
I20260808 06:07:57.816092 27310 tablet_replica.cc:330] stopping tablet replica
I20260808 06:07:57.816398 27310 raft_consensus.cc:2229] T 9d4b6dabf64a4324a7e4b837855a1bcb P d0e15846c2fc43a29763dee9c94558dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:07:57.816632 27310 raft_consensus.cc:2258] T 9d4b6dabf64a4324a7e4b837855a1bcb P d0e15846c2fc43a29763dee9c94558dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:07:57.817636 27310 ts_tablet_manager.cc:1905] T 9d4b6dabf64a4324a7e4b837855a1bcb P d0e15846c2fc43a29763dee9c94558dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:07:57.822003 27310 ts_tablet_manager.cc:1918] T 9d4b6dabf64a4324a7e4b837855a1bcb P d0e15846c2fc43a29763dee9c94558dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:07:57.822131 27310 log.cc:1196] T 9d4b6dabf64a4324a7e4b837855a1bcb P d0e15846c2fc43a29763dee9c94558dd: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/wal/wals/9d4b6dabf64a4324a7e4b837855a1bcb
I20260808 06:07:57.822448 27310 ts_tablet_manager.cc:1939] T 9d4b6dabf64a4324a7e4b837855a1bcb P d0e15846c2fc43a29763dee9c94558dd: Deleting consensus metadata
I20260808 06:07:57.824396 26971 catalog_manager.cc:4841] TS d0e15846c2fc43a29763dee9c94558dd (127.1.151.129:33379): tablet 9d4b6dabf64a4324a7e4b837855a1bcb (table test-table [id=12900096973841ceaa8a9fbe525a8ba3]) successfully deleted
W20260808 06:07:57.905793 27302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:07:57.906013 27302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:07:57.906052 27302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:07:57.909880 27302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:07:57.909931 27302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:07:57.909996 27302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.130
I20260808 06:07:57.915004 27302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:35001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.130:45151
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.151.130
--webserver_port=36177
--tserver_master_addrs=127.1.151.190:39415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.130
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:07:57.915942 27302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:07:57.916865 27302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:07:57.926035 27319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:57.926090 27302 server_base.cc:1034] running on GCE node
W20260808 06:07:57.926064 27320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:57.926054 27324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:57.926719 27302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:07:57.927292 27302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:07:57.928442 27302 hybrid_clock.cc:648] HybridClock initialized: now 1786169277928425 us; error 29 us; skew 500 ppm
Aug 08 06:07:57 dist-test-slave-zj8q krb5kdc[25996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169277, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:07:57.931023 27302 init.cc:377] Logged in from keytab as kudu/127.1.151.130@KRBTEST.COM (short username kudu)
I20260808 06:07:57.931807 27302 webserver.cc:460] Webserver started at http://127.1.151.130:36177/ using document root <none> and password file <none>
I20260808 06:07:57.932322 27302 fs_manager.cc:359] Metadata directory not provided
I20260808 06:07:57.932412 27302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:07:57.935998 27302 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260808 06:07:57.937899 27333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:57.939054 27302 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:07:57.939155 27302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/wal
uuid: "05cb149b8a664d0b89186253ff773f8b"
format_stamp: "Formatted at 2026-08-08 06:07:36 on dist-test-slave-zj8q"
server_key: "7e3281fcb12fe62372dd49fdb7451ea4"
server_key_iv: "4614d4e9c7bab6ef96db0e46cc08cd4c"
server_key_version: "encryptionkey@0"
I20260808 06:07:57.939639 27302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:07:57.957217 27302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:07:57.959925 27302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:07:57.960083 27302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:07:57.960615 27302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:07:57.961395 27302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:07:57.961442 27302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:57.961480 27302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:07:57.961500 27302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:57.970252 27302 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.130:45151
I20260808 06:07:57.970288 27512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.130:45151 every 8 connection(s)
I20260808 06:07:57.971252 27302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
I20260808 06:07:57.973058 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 27302
I20260808 06:07:57.973511 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.131:46043
--local_ip_for_outbound_sockets=127.1.151.131
--tserver_master_addrs=127.1.151.190:39415
--webserver_port=38245
--webserver_interface=127.1.151.131
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:35001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 06:07:57 dist-test-slave-zj8q krb5kdc[25996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169277, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:07:57.985064 27300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:57.973286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:59731 (local address 127.1.151.190:39415)
0808 06:07:57.973441 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:57.973446 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:57.974030 (+ 584us) server_negotiation.cc:407] Connection header received
0808 06:07:57.975711 (+ 1681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:57.975715 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:57.975779 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:57.975868 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:57.977106 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:57.977716 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:57.978929 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:57.979183 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:57.979275 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:57.981836 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:57.981839 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:57.981841 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:57.981866 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:57.983181 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:57.983648 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:57.983652 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:57.983653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:57.983728 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:57.984036 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:57.984039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:57.984041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:57.984215 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:57.984302 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:57.984765 (+ 463us) server_negotiation.cc:299] Negotiation successful
0808 06:07:57.984899 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 06:07:57.985774 27519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:57.973359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.130:59731)
0808 06:07:57.973874 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:57.973926 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:57.975427 (+ 1501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:57.976032 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:57.976061 (+ 29us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:57.976404 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:07:57.976919 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:57.976933 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:57.977859 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:57.977863 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:57.978804 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:57.978815 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:57.979336 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:57.979341 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:57.979496 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:57.980075 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:07:57.980091 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:07:57.981676 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:07:57.983290 (+ 1614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:57.983296 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:57.983309 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:57.983554 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:57.983810 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:57.983813 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:57.983815 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:57.983946 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:57.984313 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:07:57.984319 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:07:57.984544 (+ 225us) client_negotiation.cc:769] Sending connection context
0808 06:07:57.984703 (+ 159us) client_negotiation.cc:240] Negotiation successful
0808 06:07:57.985002 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":120,"threads_started":1}
I20260808 06:07:57.986915 27515 heartbeater.cc:347] Connected to a master server at 127.1.151.190:39415
I20260808 06:07:57.987157 27515 heartbeater.cc:464] Registering TS with master...
I20260808 06:07:57.987622 27515 heartbeater.cc:511] Master 127.1.151.190:39415 requested a full tablet report, sending...
I20260808 06:07:57.988658 26994 ts_manager.cc:194] Registered new tserver with Master: 05cb149b8a664d0b89186253ff773f8b (127.1.151.130:45151)
I20260808 06:07:57.989212 26994 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.130@KRBTEST.COM'} at 127.1.151.130:59731
W20260808 06:07:58.088841 27520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:07:58.089100 27520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:07:58.089159 27520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:07:58.093241 27520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:07:58.093297 27520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:07:58.093369 27520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.131
I20260808 06:07:58.098430 27520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:35001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.131:46043
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.151.131
--webserver_port=38245
--tserver_master_addrs=127.1.151.190:39415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.131
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:07:58.099364 27520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:07:58.100318 27520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:07:58.109452 27525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:07:58.109558 27531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:58.109546 27520 server_base.cc:1034] running on GCE node
W20260808 06:07:58.109469 27526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:07:58.110070 27520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:07:58.110677 27520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:07:58.111828 27520 hybrid_clock.cc:648] HybridClock initialized: now 1786169278111807 us; error 42 us; skew 500 ppm
Aug 08 06:07:58 dist-test-slave-zj8q krb5kdc[25996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169278, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:07:58.114413 27520 init.cc:377] Logged in from keytab as kudu/127.1.151.131@KRBTEST.COM (short username kudu)
I20260808 06:07:58.115201 27520 webserver.cc:460] Webserver started at http://127.1.151.131:38245/ using document root <none> and password file <none>
I20260808 06:07:58.115718 27520 fs_manager.cc:359] Metadata directory not provided
I20260808 06:07:58.115797 27520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:07:58.119509 27520 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:07:58.121485 27542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:58.122690 27520 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:07:58.122794 27520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/wal
uuid: "e9e76571d8944ab7a9226db3a83a8719"
format_stamp: "Formatted at 2026-08-08 06:07:36 on dist-test-slave-zj8q"
server_key: "54b70e21e3349c4ac35822ad6ed91275"
server_key_iv: "fe06e4a73ea64348933d25d7624ccf39"
server_key_version: "encryptionkey@0"
I20260808 06:07:58.123346 27520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:07:58.162853 27520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:07:58.165982 27520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:07:58.166203 27520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:07:58.166826 27520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:07:58.167660 27520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:07:58.167707 27520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:58.167761 27520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:07:58.167788 27520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:07:58.176342 27520 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.131:46043
I20260808 06:07:58.176379 27724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.131:46043 every 8 connection(s)
I20260808 06:07:58.177289 27520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
I20260808 06:07:58.181875 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 27520
Aug 08 06:07:58 dist-test-slave-zj8q krb5kdc[25996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169278, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:07:58.191232 27732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:58.182668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53300 (local address 127.1.151.190:39415)
0808 06:07:58.182911 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:58.182916 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:58.183079 (+ 163us) server_negotiation.cc:407] Connection header received
0808 06:07:58.183176 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:58.183179 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:58.183225 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:58.183312 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:58.184339 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:58.184837 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:58.185995 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:58.186310 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:58.186387 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:58.187745 (+ 1358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:58.187749 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:58.187752 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:58.187778 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:58.189422 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:58.189861 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:58.189864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:58.189866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:58.189950 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:58.190258 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:58.190260 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:58.190262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:58.190485 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:58.190659 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:58.190867 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 06:07:58.191032 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":74,"threads_started":1}
I20260808 06:07:58.191716 27300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:58.179111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:33419 (local address 127.1.151.190:39415)
0808 06:07:58.179250 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:58.179254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:58.180057 (+ 803us) server_negotiation.cc:407] Connection header received
0808 06:07:58.181337 (+ 1280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:58.181340 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:58.181387 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:58.181458 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:58.182960 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:58.183627 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:58.185044 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:58.185258 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:58.185332 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:58.188064 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:58.188068 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:58.188070 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:58.188104 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:58.189478 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:58.190043 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:58.190047 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:58.190048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:58.190171 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:58.190593 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:58.190596 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:58.190598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:58.190800 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:58.190887 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:58.191474 (+ 587us) server_negotiation.cc:299] Negotiation successful
0808 06:07:58.191580 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 06:07:58.192687 27729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:58.179476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.131:33419)
0808 06:07:58.179907 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:58.179949 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:58.181130 (+ 1181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:58.181615 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:58.181624 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:58.182114 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:07:58.182768 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:58.182790 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:58.183773 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:58.183777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:58.184926 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:58.184939 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:58.185404 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:58.185408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:58.185627 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:58.186249 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:07:58.186266 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:07:58.187893 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:07:58.189639 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:58.189644 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:58.189655 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:58.189935 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:58.190256 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:07:58.190259 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:07:58.190261 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:07:58.190446 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:07:58.190902 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:07:58.190909 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:07:58.191229 (+ 320us) client_negotiation.cc:769] Sending connection context
0808 06:07:58.191436 (+ 207us) client_negotiation.cc:240] Negotiation successful
0808 06:07:58.191775 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":108,"threads_started":1}
I20260808 06:07:58.193706 27726 heartbeater.cc:347] Connected to a master server at 127.1.151.190:39415
I20260808 06:07:58.193914 27726 heartbeater.cc:464] Registering TS with master...
I20260808 06:07:58.194404 27726 heartbeater.cc:511] Master 127.1.151.190:39415 requested a full tablet report, sending...
I20260808 06:07:58.195413 26994 ts_manager.cc:194] Registered new tserver with Master: e9e76571d8944ab7a9226db3a83a8719 (127.1.151.131:46043)
I20260808 06:07:58.195937 26994 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.131@KRBTEST.COM'} at 127.1.151.131:33419
I20260808 06:07:58.204698 1630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 06:07:58.208959 27300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:58.205410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53316 (local address 127.1.151.190:39415)
0808 06:07:58.205602 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:58.205606 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:58.205723 (+ 117us) server_negotiation.cc:407] Connection header received
0808 06:07:58.205874 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:58.205877 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:58.205977 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:58.206062 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:07:58.206342 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:58.206895 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:58.207909 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:58.208138 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:58.208219 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:58.208465 (+ 246us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:07:58.208649 (+ 184us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 06:07:58.208855 (+ 206us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53316: token expired
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 06:07:58.209038 27300 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53316: token expired
W20260808 06:07:58.209438 26875 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.1.151.190:39415)
I20260808 06:07:58.215855 27300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:58.209765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53320 (local address 127.1.151.190:39415)
0808 06:07:58.209865 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:58.209868 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:58.209879 (+ 11us) server_negotiation.cc:407] Connection header received
0808 06:07:58.209957 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:58.209959 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:58.209994 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:58.210071 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:07:58.210761 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:58.211244 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:58.211988 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:58.212208 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:58.212278 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:58.213382 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:07:58.213385 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:07:58.213387 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:07:58.213408 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:07:58.214673 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:58.215010 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:58.215012 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:58.215013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:58.215096 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:07:58.215272 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:07:58.215274 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:07:58.215275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:07:58.215417 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:07:58.215512 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:58.215673 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 06:07:58.215754 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 06:07:58.218014 26994 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53320:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 06:07:58.221371 26994 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 06:07:58.228863 27194 tablet_service.cc:1467] Processing CreateTablet for tablet ee92ac18a1b24b11afdbed8e566767ae (DEFAULT_TABLE table=test-table [id=9c8785d875574e79845554e123eb409d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:07:58.229571 27194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee92ac18a1b24b11afdbed8e566767ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:07:58.232419 27123 tablet_bootstrap.cc:492] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd: Bootstrap starting.
I20260808 06:07:58.232838 27307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:58.227766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:46043 (local address 127.0.0.1:53300)
0808 06:07:58.227865 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:58.227879 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:58.227952 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:58.229138 (+ 1186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:58.229140 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:58.229157 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:07:58.229231 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:58.229236 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:58.230498 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:58.230501 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:58.231277 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:58.231287 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:58.232398 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:58.232402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:58.232511 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:58.232578 (+ 67us) client_negotiation.cc:769] Sending connection context
0808 06:07:58.232634 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 06:07:58.232717 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 06:07:58.233299 27123 tablet_bootstrap.cc:654] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd: Neither blocks nor log segments found. Creating new log.
I20260808 06:07:58.233573 27745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:58.228127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53300 (local address 127.1.151.131:46043)
0808 06:07:58.228608 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:58.228621 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:58.228653 (+ 32us) server_negotiation.cc:407] Connection header received
0808 06:07:58.228734 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:58.228740 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:58.228970 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:58.229136 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:07:58.229305 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:58.230392 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:58.231409 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:58.232255 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:58.233107 (+ 852us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:58.233268 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:58.233323 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 06:07:58.233420 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":113,"threads_started":1}
I20260808 06:07:58.234807 27123 tablet_bootstrap.cc:492] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd: No bootstrap required, opened a new log
I20260808 06:07:58.234890 27123 ts_tablet_manager.cc:1397] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260808 06:07:58.235275 27123 raft_consensus.cc:348] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e76571d8944ab7a9226db3a83a8719" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 46043 } } peers { permanent_uuid: "05cb149b8a664d0b89186253ff773f8b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45151 } } peers { permanent_uuid: "d0e15846c2fc43a29763dee9c94558dd" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 33379 } }
I20260808 06:07:58.235373 27123 raft_consensus.cc:374] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:07:58.235409 27123 raft_consensus.cc:729] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0e15846c2fc43a29763dee9c94558dd, State: Initialized, Role: FOLLOWER
I20260808 06:07:58.235524 27123 consensus_queue.cc:260] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e76571d8944ab7a9226db3a83a8719" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 46043 } } peers { permanent_uuid: "05cb149b8a664d0b89186253ff773f8b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45151 } } peers { permanent_uuid: "d0e15846c2fc43a29763dee9c94558dd" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 33379 } }
I20260808 06:07:58.236532 27123 ts_tablet_manager.cc:1428] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 06:07:58.237236 27744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:58.228081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:45151 (local address 127.0.0.1:37860)
0808 06:07:58.228512 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:58.228525 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:58.228602 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:58.233223 (+ 4621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:58.233226 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:58.233247 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:07:58.233336 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:58.233343 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:58.234444 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:58.234447 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:58.235573 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:58.235585 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:58.236821 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:58.236824 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:58.236922 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:58.236945 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 06:07:58.237021 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 06:07:58.237101 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":96,"threads_started":1}
I20260808 06:07:58.236770 27623 tablet_service.cc:1467] Processing CreateTablet for tablet ee92ac18a1b24b11afdbed8e566767ae (DEFAULT_TABLE table=test-table [id=9c8785d875574e79845554e123eb409d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:07:58.237337 27746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:58.228246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37860 (local address 127.1.151.130:45151)
0808 06:07:58.232738 (+ 4492us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:58.232750 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:58.232773 (+ 23us) server_negotiation.cc:407] Connection header received
0808 06:07:58.232848 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:58.232853 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:58.233070 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:58.233230 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:07:58.233438 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:58.234309 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:58.235716 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:58.236515 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:58.236617 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:58.236749 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:58.237087 (+ 338us) server_negotiation.cc:299] Negotiation successful
0808 06:07:58.237200 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4382,"thread_start_us":134,"threads_started":1}
I20260808 06:07:58.237893 27623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee92ac18a1b24b11afdbed8e566767ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:07:58.239748 27417 tablet_service.cc:1467] Processing CreateTablet for tablet ee92ac18a1b24b11afdbed8e566767ae (DEFAULT_TABLE table=test-table [id=9c8785d875574e79845554e123eb409d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:07:58.240644 27417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee92ac18a1b24b11afdbed8e566767ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:07:58.245111 27749 tablet_bootstrap.cc:492] T ee92ac18a1b24b11afdbed8e566767ae P e9e76571d8944ab7a9226db3a83a8719: Bootstrap starting.
I20260808 06:07:58.247195 27750 tablet_bootstrap.cc:492] T ee92ac18a1b24b11afdbed8e566767ae P 05cb149b8a664d0b89186253ff773f8b: Bootstrap starting.
I20260808 06:07:58.247802 27749 tablet_bootstrap.cc:654] T ee92ac18a1b24b11afdbed8e566767ae P e9e76571d8944ab7a9226db3a83a8719: Neither blocks nor log segments found. Creating new log.
I20260808 06:07:58.248476 27749 log.cc:824] T ee92ac18a1b24b11afdbed8e566767ae P e9e76571d8944ab7a9226db3a83a8719: Log is configured to *not* fsync() on all Append() calls
I20260808 06:07:58.249470 27750 tablet_bootstrap.cc:654] T ee92ac18a1b24b11afdbed8e566767ae P 05cb149b8a664d0b89186253ff773f8b: Neither blocks nor log segments found. Creating new log.
I20260808 06:07:58.250136 27750 log.cc:824] T ee92ac18a1b24b11afdbed8e566767ae P 05cb149b8a664d0b89186253ff773f8b: Log is configured to *not* fsync() on all Append() calls
I20260808 06:07:58.250420 27749 tablet_bootstrap.cc:492] T ee92ac18a1b24b11afdbed8e566767ae P e9e76571d8944ab7a9226db3a83a8719: No bootstrap required, opened a new log
I20260808 06:07:58.250618 27749 ts_tablet_manager.cc:1397] T ee92ac18a1b24b11afdbed8e566767ae P e9e76571d8944ab7a9226db3a83a8719: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 06:07:58.251818 27750 tablet_bootstrap.cc:492] T ee92ac18a1b24b11afdbed8e566767ae P 05cb149b8a664d0b89186253ff773f8b: No bootstrap required, opened a new log
I20260808 06:07:58.252007 27750 ts_tablet_manager.cc:1397] T ee92ac18a1b24b11afdbed8e566767ae P 05cb149b8a664d0b89186253ff773f8b: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260808 06:07:58.253441 27749 raft_consensus.cc:348] T ee92ac18a1b24b11afdbed8e566767ae P e9e76571d8944ab7a9226db3a83a8719 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e76571d8944ab7a9226db3a83a8719" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 46043 } } peers { permanent_uuid: "05cb149b8a664d0b89186253ff773f8b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45151 } } peers { permanent_uuid: "d0e15846c2fc43a29763dee9c94558dd" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 33379 } }
I20260808 06:07:58.253659 27749 raft_consensus.cc:374] T ee92ac18a1b24b11afdbed8e566767ae P e9e76571d8944ab7a9226db3a83a8719 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:07:58.253701 27749 raft_consensus.cc:729] T ee92ac18a1b24b11afdbed8e566767ae P e9e76571d8944ab7a9226db3a83a8719 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9e76571d8944ab7a9226db3a83a8719, State: Initialized, Role: FOLLOWER
I20260808 06:07:58.254096 27749 consensus_queue.cc:260] T ee92ac18a1b24b11afdbed8e566767ae P e9e76571d8944ab7a9226db3a83a8719 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e76571d8944ab7a9226db3a83a8719" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 46043 } } peers { permanent_uuid: "05cb149b8a664d0b89186253ff773f8b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45151 } } peers { permanent_uuid: "d0e15846c2fc43a29763dee9c94558dd" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 33379 } }
I20260808 06:07:58.254725 27726 heartbeater.cc:503] Master 127.1.151.190:39415 was elected leader, sending a full tablet report...
I20260808 06:07:58.254896 27750 raft_consensus.cc:348] T ee92ac18a1b24b11afdbed8e566767ae P 05cb149b8a664d0b89186253ff773f8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e76571d8944ab7a9226db3a83a8719" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 46043 } } peers { permanent_uuid: "05cb149b8a664d0b89186253ff773f8b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45151 } } peers { permanent_uuid: "d0e15846c2fc43a29763dee9c94558dd" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 33379 } }
I20260808 06:07:58.255092 27749 ts_tablet_manager.cc:1428] T ee92ac18a1b24b11afdbed8e566767ae P e9e76571d8944ab7a9226db3a83a8719: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 06:07:58.255100 27750 raft_consensus.cc:374] T ee92ac18a1b24b11afdbed8e566767ae P 05cb149b8a664d0b89186253ff773f8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:07:58.255151 27750 raft_consensus.cc:729] T ee92ac18a1b24b11afdbed8e566767ae P 05cb149b8a664d0b89186253ff773f8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05cb149b8a664d0b89186253ff773f8b, State: Initialized, Role: FOLLOWER
I20260808 06:07:58.255527 27750 consensus_queue.cc:260] T ee92ac18a1b24b11afdbed8e566767ae P 05cb149b8a664d0b89186253ff773f8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e76571d8944ab7a9226db3a83a8719" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 46043 } } peers { permanent_uuid: "05cb149b8a664d0b89186253ff773f8b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45151 } } peers { permanent_uuid: "d0e15846c2fc43a29763dee9c94558dd" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 33379 } }
I20260808 06:07:58.256342 27750 ts_tablet_manager.cc:1428] T ee92ac18a1b24b11afdbed8e566767ae P 05cb149b8a664d0b89186253ff773f8b: Time spent starting tablet: real 0.004s user 0.004s sys 0.001s
I20260808 06:07:58.256736 27515 heartbeater.cc:503] Master 127.1.151.190:39415 was elected leader, sending a full tablet report...
I20260808 06:07:58.257287 27753 raft_consensus.cc:482] T ee92ac18a1b24b11afdbed8e566767ae P e9e76571d8944ab7a9226db3a83a8719 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:07:58.257426 27753 raft_consensus.cc:504] T ee92ac18a1b24b11afdbed8e566767ae P e9e76571d8944ab7a9226db3a83a8719 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e76571d8944ab7a9226db3a83a8719" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 46043 } } peers { permanent_uuid: "05cb149b8a664d0b89186253ff773f8b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45151 } } peers { permanent_uuid: "d0e15846c2fc43a29763dee9c94558dd" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 33379 } }
I20260808 06:07:58.258580 27753 leader_election.cc:288] T ee92ac18a1b24b11afdbed8e566767ae P e9e76571d8944ab7a9226db3a83a8719 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05cb149b8a664d0b89186253ff773f8b (127.1.151.130:45151), d0e15846c2fc43a29763dee9c94558dd (127.1.151.129:33379)
I20260808 06:07:58.263345 27729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:58.258721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:45151 (local address 127.1.151.131:53745)
0808 06:07:58.258938 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:58.258954 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:58.259031 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:58.259455 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:58.259457 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:58.259475 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:07:58.259605 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:58.259612 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:58.260683 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:58.260687 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:58.261984 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:58.261995 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:58.262929 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:58.262933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:58.263018 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:58.263044 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 06:07:58.263141 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 06:07:58.263213 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"spinlock_wait_cycles":24960}
I20260808 06:07:58.263687 27746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:58.259171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:53745 (local address 127.1.151.130:45151)
0808 06:07:58.259269 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:58.259273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:58.259284 (+ 11us) server_negotiation.cc:407] Connection header received
0808 06:07:58.259320 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:58.259322 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:58.259357 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:58.259417 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:07:58.259712 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:58.260434 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:58.262126 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:58.262793 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:58.262942 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:58.262991 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:58.263503 (+ 512us) server_negotiation.cc:299] Negotiation successful
0808 06:07:58.263579 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 06:07:58.263969 27309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:58.259103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:55399 (local address 127.1.151.129:33379)
0808 06:07:58.259242 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:58.259247 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:58.259463 (+ 216us) server_negotiation.cc:407] Connection header received
0808 06:07:58.259648 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:58.259651 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:58.259713 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:58.259801 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:07:58.260001 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:58.260791 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:58.262257 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:58.262969 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:58.263081 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:58.263238 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:58.263781 (+ 543us) server_negotiation.cc:299] Negotiation successful
0808 06:07:58.263871 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 06:07:58.264086 27755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:58.258755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:33379 (local address 127.1.151.131:55399)
0808 06:07:58.259387 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:58.259398 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 06:07:58.259533 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:07:58.259813 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:07:58.259815 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:07:58.259830 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:07:58.259912 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:58.259916 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:58.260958 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:58.260961 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:58.262132 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:07:58.262141 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:07:58.263078 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:07:58.263082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:07:58.263605 (+ 523us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:58.263628 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 06:07:58.263703 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 06:07:58.263949 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"spinlock_wait_cycles":2688,"thread_start_us":99,"threads_started":1}
I20260808 06:07:58.264191 27446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee92ac18a1b24b11afdbed8e566767ae" candidate_uuid: "e9e76571d8944ab7a9226db3a83a8719" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05cb149b8a664d0b89186253ff773f8b" is_pre_election: true
I20260808 06:07:58.264443 27446 raft_consensus.cc:2454] T ee92ac18a1b24b11afdbed8e566767ae P 05cb149b8a664d0b89186253ff773f8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9e76571d8944ab7a9226db3a83a8719 in term 0.
I20260808 06:07:58.264623 27227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee92ac18a1b24b11afdbed8e566767ae" candidate_uuid: "e9e76571d8944ab7a9226db3a83a8719" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e15846c2fc43a29763dee9c94558dd" is_pre_election: true
I20260808 06:07:58.264879 27227 raft_consensus.cc:2454] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9e76571d8944ab7a9226db3a83a8719 in term 0.
I20260808 06:07:58.264918 27551 leader_election.cc:302] T ee92ac18a1b24b11afdbed8e566767ae P e9e76571d8944ab7a9226db3a83a8719 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05cb149b8a664d0b89186253ff773f8b, e9e76571d8944ab7a9226db3a83a8719; no voters:
I20260808 06:07:58.265163 27753 raft_consensus.cc:2781] T ee92ac18a1b24b11afdbed8e566767ae P e9e76571d8944ab7a9226db3a83a8719 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 06:07:58.265247 27753 raft_consensus.cc:482] T ee92ac18a1b24b11afdbed8e566767ae P e9e76571d8944ab7a9226db3a83a8719 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 06:07:58.265298 27753 raft_consensus.cc:3037] T ee92ac18a1b24b11afdbed8e566767ae P e9e76571d8944ab7a9226db3a83a8719 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:07:58.266307 27753 raft_consensus.cc:504] T ee92ac18a1b24b11afdbed8e566767ae P e9e76571d8944ab7a9226db3a83a8719 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e76571d8944ab7a9226db3a83a8719" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 46043 } } peers { permanent_uuid: "05cb149b8a664d0b89186253ff773f8b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45151 } } peers { permanent_uuid: "d0e15846c2fc43a29763dee9c94558dd" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 33379 } }
I20260808 06:07:58.266618 27753 leader_election.cc:288] T ee92ac18a1b24b11afdbed8e566767ae P e9e76571d8944ab7a9226db3a83a8719 [CANDIDATE]: Term 1 election: Requested vote from peers 05cb149b8a664d0b89186253ff773f8b (127.1.151.130:45151), d0e15846c2fc43a29763dee9c94558dd (127.1.151.129:33379)
I20260808 06:07:58.266927 27446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee92ac18a1b24b11afdbed8e566767ae" candidate_uuid: "e9e76571d8944ab7a9226db3a83a8719" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05cb149b8a664d0b89186253ff773f8b"
I20260808 06:07:58.266969 27227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee92ac18a1b24b11afdbed8e566767ae" candidate_uuid: "e9e76571d8944ab7a9226db3a83a8719" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e15846c2fc43a29763dee9c94558dd"
I20260808 06:07:58.267022 27446 raft_consensus.cc:3037] T ee92ac18a1b24b11afdbed8e566767ae P 05cb149b8a664d0b89186253ff773f8b [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:07:58.267092 27227 raft_consensus.cc:3037] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:07:58.267906 27446 raft_consensus.cc:2454] T ee92ac18a1b24b11afdbed8e566767ae P 05cb149b8a664d0b89186253ff773f8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9e76571d8944ab7a9226db3a83a8719 in term 1.
I20260808 06:07:58.268189 27551 leader_election.cc:302] T ee92ac18a1b24b11afdbed8e566767ae P e9e76571d8944ab7a9226db3a83a8719 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05cb149b8a664d0b89186253ff773f8b, e9e76571d8944ab7a9226db3a83a8719; no voters:
I20260808 06:07:58.268229 27227 raft_consensus.cc:2454] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9e76571d8944ab7a9226db3a83a8719 in term 1.
I20260808 06:07:58.268340 27753 raft_consensus.cc:2781] T ee92ac18a1b24b11afdbed8e566767ae P e9e76571d8944ab7a9226db3a83a8719 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:07:58.268535 27753 raft_consensus.cc:686] T ee92ac18a1b24b11afdbed8e566767ae P e9e76571d8944ab7a9226db3a83a8719 [term 1 LEADER]: Becoming Leader. State: Replica: e9e76571d8944ab7a9226db3a83a8719, State: Running, Role: LEADER
I20260808 06:07:58.268826 27753 consensus_queue.cc:237] T ee92ac18a1b24b11afdbed8e566767ae P e9e76571d8944ab7a9226db3a83a8719 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e76571d8944ab7a9226db3a83a8719" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 46043 } } peers { permanent_uuid: "05cb149b8a664d0b89186253ff773f8b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45151 } } peers { permanent_uuid: "d0e15846c2fc43a29763dee9c94558dd" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 33379 } }
I20260808 06:07:58.271632 26994 catalog_manager.cc:5495] T ee92ac18a1b24b11afdbed8e566767ae P e9e76571d8944ab7a9226db3a83a8719 reported cstate change: term changed from 0 to 1, leader changed from <none> to e9e76571d8944ab7a9226db3a83a8719 (127.1.151.131). New cstate: current_term: 1 leader_uuid: "e9e76571d8944ab7a9226db3a83a8719" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e76571d8944ab7a9226db3a83a8719" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 46043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05cb149b8a664d0b89186253ff773f8b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0e15846c2fc43a29763dee9c94558dd" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 33379 } health_report { overall_health: UNKNOWN } } }
W20260808 06:07:58.278112 27296 tablet.cc:2369] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 06:07:58.289526 27745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:58.286211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53302 (local address 127.1.151.131:46043)
0808 06:07:58.286337 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 06:07:58.286341 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:07:58.286376 (+ 35us) server_negotiation.cc:407] Connection header received
0808 06:07:58.286526 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:07:58.286528 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:07:58.286582 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:07:58.286689 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:07:58.286943 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:58.287438 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:58.288229 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:07:58.288427 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:07:58.288495 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:07:58.288765 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:07:58.289147 (+ 382us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:07:58.289233 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 06:07:58.289335 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 06:07:58.289395 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 06:07:58.295487 27227 raft_consensus.cc:1261] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [term 1 FOLLOWER]: Refusing update from remote peer e9e76571d8944ab7a9226db3a83a8719: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:07:58.296198 27446 raft_consensus.cc:1261] T ee92ac18a1b24b11afdbed8e566767ae P 05cb149b8a664d0b89186253ff773f8b [term 1 FOLLOWER]: Refusing update from remote peer e9e76571d8944ab7a9226db3a83a8719: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:07:58.296169 27753 consensus_queue.cc:1035] T ee92ac18a1b24b11afdbed8e566767ae P e9e76571d8944ab7a9226db3a83a8719 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0e15846c2fc43a29763dee9c94558dd" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 33379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:07:58.296751 27753 consensus_queue.cc:1035] T ee92ac18a1b24b11afdbed8e566767ae P e9e76571d8944ab7a9226db3a83a8719 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05cb149b8a664d0b89186253ff773f8b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:07:58.301126 27761 mvcc.cc:205] Tried to move back new op lower bound from 7316149363890880512 to 7316149363791593472. Current Snapshot: MvccSnapshot[applied={T|T < 7316149363890880512}]
I20260808 06:07:58.305160 27766 mvcc.cc:205] Tried to move back new op lower bound from 7316149363890880512 to 7316149363791593472. Current Snapshot: MvccSnapshot[applied={T|T < 7316149363890880512}]
I20260808 06:07:58.311264 26994 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:07:58.311439 26994 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:07:58.313795 26994 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579: Sending DeleteTablet for 3 replicas of tablet ee92ac18a1b24b11afdbed8e566767ae
I20260808 06:07:58.314446 27194 tablet_service.cc:1514] Processing DeleteTablet for tablet ee92ac18a1b24b11afdbed8e566767ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:07:58 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:55392
I20260808 06:07:58.314548 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 27061
I20260808 06:07:58.314625 27310 tablet_replica.cc:330] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd: stopping tablet replica
I20260808 06:07:58.314595 27417 tablet_service.cc:1514] Processing DeleteTablet for tablet ee92ac18a1b24b11afdbed8e566767ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:07:58 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:37860
I20260808 06:07:58.314734 27310 raft_consensus.cc:2229] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:07:58.314848 27310 raft_consensus.cc:2258] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:07:58.322502 27782 tablet_replica.cc:330] T ee92ac18a1b24b11afdbed8e566767ae P 05cb149b8a664d0b89186253ff773f8b: stopping tablet replica
I20260808 06:07:58.322842 27782 raft_consensus.cc:2229] T ee92ac18a1b24b11afdbed8e566767ae P 05cb149b8a664d0b89186253ff773f8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:07:58.323140 27782 raft_consensus.cc:2258] T ee92ac18a1b24b11afdbed8e566767ae P 05cb149b8a664d0b89186253ff773f8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:07:58.324903 27623 tablet_service.cc:1514] Processing DeleteTablet for tablet ee92ac18a1b24b11afdbed8e566767ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:07:58 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:53300
I20260808 06:07:58.325553 27783 tablet_replica.cc:330] T ee92ac18a1b24b11afdbed8e566767ae P e9e76571d8944ab7a9226db3a83a8719: stopping tablet replica
I20260808 06:07:58.325941 27783 raft_consensus.cc:2229] T ee92ac18a1b24b11afdbed8e566767ae P e9e76571d8944ab7a9226db3a83a8719 [term 1 LEADER]: Raft consensus shutting down.
I20260808 06:07:58.326355 27783 raft_consensus.cc:2258] T ee92ac18a1b24b11afdbed8e566767ae P e9e76571d8944ab7a9226db3a83a8719 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:07:58.328302 27783 ts_tablet_manager.cc:1905] T ee92ac18a1b24b11afdbed8e566767ae P e9e76571d8944ab7a9226db3a83a8719: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:07:58.328435 27782 ts_tablet_manager.cc:1905] T ee92ac18a1b24b11afdbed8e566767ae P 05cb149b8a664d0b89186253ff773f8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:07:58.330760 27783 ts_tablet_manager.cc:1918] T ee92ac18a1b24b11afdbed8e566767ae P e9e76571d8944ab7a9226db3a83a8719: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:07:58.330847 27783 log.cc:1196] T ee92ac18a1b24b11afdbed8e566767ae P e9e76571d8944ab7a9226db3a83a8719: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/wal/wals/ee92ac18a1b24b11afdbed8e566767ae
I20260808 06:07:58.331161 27783 ts_tablet_manager.cc:1939] T ee92ac18a1b24b11afdbed8e566767ae P e9e76571d8944ab7a9226db3a83a8719: Deleting consensus metadata
I20260808 06:07:58.331856 27782 ts_tablet_manager.cc:1918] T ee92ac18a1b24b11afdbed8e566767ae P 05cb149b8a664d0b89186253ff773f8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:07:58.331969 27782 log.cc:1196] T ee92ac18a1b24b11afdbed8e566767ae P 05cb149b8a664d0b89186253ff773f8b: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/wal/wals/ee92ac18a1b24b11afdbed8e566767ae
I20260808 06:07:58.332274 26971 catalog_manager.cc:4841] TS e9e76571d8944ab7a9226db3a83a8719 (127.1.151.131:46043): tablet ee92ac18a1b24b11afdbed8e566767ae (table test-table [id=9c8785d875574e79845554e123eb409d]) successfully deleted
I20260808 06:07:58.332461 27782 ts_tablet_manager.cc:1939] T ee92ac18a1b24b11afdbed8e566767ae P 05cb149b8a664d0b89186253ff773f8b: Deleting consensus metadata
I20260808 06:07:58.333747 26973 catalog_manager.cc:4841] TS 05cb149b8a664d0b89186253ff773f8b (127.1.151.130:45151): tablet ee92ac18a1b24b11afdbed8e566767ae (table test-table [id=9c8785d875574e79845554e123eb409d]) successfully deleted
W20260808 06:07:58.339428 26971 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.151.129:33379 (error 108)
I20260808 06:07:58.340018 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 27302
I20260808 06:07:58.340798 27744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:07:58.340240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:33379 (local address 127.0.0.1:55406)
0808 06:07:58.340632 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0808 06:07:58.340706 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.129:33379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299}
W20260808 06:07:58.347095 26971 catalog_manager.cc:4584] TS d0e15846c2fc43a29763dee9c94558dd (127.1.151.129:33379): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ee92ac18a1b24b11afdbed8e566767ae: Network error: Client connection negotiation failed: client connection to 127.1.151.129:33379: connect: Connection refused (error 111)
I20260808 06:07:58.353682 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 27520
I20260808 06:07:58.366569 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 26937
I20260808 06:08:18.379267 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:39415
--webserver_interface=127.1.151.190
--webserver_port=41189
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:35001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:39415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:08:18.492744 27785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:18.492990 27785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:18.493027 27785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:18.496938 27785 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 06:08:18.496982 27785 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 06:08:18.496996 27785 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 06:08:18.497009 27785 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:08:18.497025 27785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:18.497041 27785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:18.497061 27785 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:08:18.497078 27785 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:08:18.502234 27785 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:35001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:39415
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:39415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=41189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:18.503212 27785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:18.504156 27785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 06:08:18.512534 27785 server_base.cc:1034] running on GCE node
W20260808 06:08:18.512501 27792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:18.512494 27794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:18.512555 27798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:18.513294 27785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:18.514356 27785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:18.515527 27785 hybrid_clock.cc:648] HybridClock initialized: now 1786169298515492 us; error 43 us; skew 500 ppm
Aug 08 06:08:18 dist-test-slave-zj8q krb5kdc[25996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169298, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:08:18.518173 27785 init.cc:377] Logged in from keytab as kudu/127.1.151.190@KRBTEST.COM (short username kudu)
I20260808 06:08:18.518991 27785 webserver.cc:460] Webserver started at http://127.1.151.190:41189/ using document root <none> and password file <none>
I20260808 06:08:18.519495 27785 fs_manager.cc:359] Metadata directory not provided
I20260808 06:08:18.519553 27785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:08:18.523571 27785 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 06:08:18.525612 27812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:18.526841 27785 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:08:18.526938 27785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "1d12e98c2324449b8fbf2a5e12396579"
format_stamp: "Formatted at 2026-08-08 06:07:35 on dist-test-slave-zj8q"
server_key: "c15b5a4cf93da8907c0fddcdc9eaa2ca"
server_key_iv: "8f3e1e1f4fe612d95af6b4a87993804d"
server_key_version: "encryptionkey@0"
I20260808 06:08:18.527446 27785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:08:18.546808 27785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:08:18.549620 27785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:08:18.549768 27785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:08:18.556759 27785 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:39415
I20260808 06:08:18.557052 27902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:39415 every 8 connection(s)
I20260808 06:08:18.557879 27785 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:08:18.562459 27904 sys_catalog.cc:263] Verifying existing consensus state
I20260808 06:08:18.564168 27904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579: Bootstrap starting.
I20260808 06:08:18.567358 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 27785
I20260808 06:08:18.567777 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.129:33379
--local_ip_for_outbound_sockets=127.1.151.129
--tserver_master_addrs=127.1.151.190:39415
--webserver_port=46077
--webserver_interface=127.1.151.129
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:35001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 06:08:18.582902 27904 log.cc:824] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579: Log is configured to *not* fsync() on all Append() calls
I20260808 06:08:18.594590 27904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 06:08:18.594944 27904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579: Bootstrap complete.
I20260808 06:08:18.599267 27904 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d12e98c2324449b8fbf2a5e12396579" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39415 } }
I20260808 06:08:18.599653 27904 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d12e98c2324449b8fbf2a5e12396579, State: Initialized, Role: FOLLOWER
I20260808 06:08:18.600123 27904 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d12e98c2324449b8fbf2a5e12396579" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39415 } }
I20260808 06:08:18.600253 27904 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:08:18.600319 27904 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:08:18.600414 27904 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [term 2 FOLLOWER]: Advancing to term 3
I20260808 06:08:18.602267 27904 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d12e98c2324449b8fbf2a5e12396579" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39415 } }
I20260808 06:08:18.602576 27904 leader_election.cc:302] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d12e98c2324449b8fbf2a5e12396579; no voters:
I20260808 06:08:18.602876 27904 leader_election.cc:288] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 06:08:18.602979 27910 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 06:08:18.603179 27910 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [term 3 LEADER]: Becoming Leader. State: Replica: 1d12e98c2324449b8fbf2a5e12396579, State: Running, Role: LEADER
I20260808 06:08:18.603541 27910 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d12e98c2324449b8fbf2a5e12396579" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39415 } }
I20260808 06:08:18.604125 27904 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:08:18.604491 27910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d12e98c2324449b8fbf2a5e12396579. Latest consensus state: current_term: 3 leader_uuid: "1d12e98c2324449b8fbf2a5e12396579" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d12e98c2324449b8fbf2a5e12396579" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39415 } } }
I20260808 06:08:18.604709 27910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [sys.catalog]: This master's current role is: LEADER
I20260808 06:08:18.604890 27912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "1d12e98c2324449b8fbf2a5e12396579" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d12e98c2324449b8fbf2a5e12396579" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39415 } } }
I20260808 06:08:18.605077 27912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579 [sys.catalog]: This master's current role is: LEADER
I20260808 06:08:18.605155 27920 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:08:18.610821 27920 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:08:18.611528 27920 catalog_manager.cc:1259] Loaded cluster ID: 2b628fe1aedf4d5bbd03ad57895d53a4
I20260808 06:08:18.611595 27920 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:08:18.614107 27920 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:08:18.614894 27920 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579: Loaded TSK: 0
I20260808 06:08:18.614984 27920 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1d12e98c2324449b8fbf2a5e12396579: Loaded TSK: 1
F20260808 06:08:18.615744 27920 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f4ea02c49ad google::LogMessage::Fail() at ??:0
@ 0x7f4ea02c788a google::LogMessage::SendToLog() at ??:0
@ 0x7f4ea02c44de google::LogMessage::Flush() at ??:0
@ 0x7f4ea02c5a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f4ea2f8818d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f4ea2f80f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f4eaac297bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f4eaac0399c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f4eaac04178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f4eaac45df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f4eaac6482b std::function<>::operator()() at ??:0
@ 0x7f4eaac03b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f4eaac04f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f4eaac01579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f4eaac45640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f4ea25163d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f4ea2516c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f4ea25183f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f4ea2507512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f4ea3b4b6db start_thread at ??:0
@ 0x7f4e9edc261f clone at ??:0
W20260808 06:08:18.705893 27908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:18.706279 27908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:18.706359 27908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:18.712369 27908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:18.712442 27908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:18.712530 27908 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.129
I20260808 06:08:18.720335 27908 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:35001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.129:33379
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.151.129
--webserver_port=46077
--tserver_master_addrs=127.1.151.190:39415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.129
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:18.721729 27908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:18.723134 27908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:08:18.734207 27984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:18.734207 27988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:18.734483 27908 server_base.cc:1034] running on GCE node
W20260808 06:08:18.734629 27992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:18.735008 27908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:18.735620 27908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:18.736788 27908 hybrid_clock.cc:648] HybridClock initialized: now 1786169298736756 us; error 46 us; skew 500 ppm
Aug 08 06:08:18 dist-test-slave-zj8q krb5kdc[25996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169298, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:08:18.739440 27908 init.cc:377] Logged in from keytab as kudu/127.1.151.129@KRBTEST.COM (short username kudu)
I20260808 06:08:18.740295 27908 webserver.cc:460] Webserver started at http://127.1.151.129:46077/ using document root <none> and password file <none>
I20260808 06:08:18.740828 27908 fs_manager.cc:359] Metadata directory not provided
I20260808 06:08:18.740890 27908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:08:18.744786 27908 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 06:08:18.747035 28008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:18.748476 27908 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 06:08:18.748587 27908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/wal
uuid: "d0e15846c2fc43a29763dee9c94558dd"
format_stamp: "Formatted at 2026-08-08 06:07:36 on dist-test-slave-zj8q"
server_key: "995db3cfdead3e101bee455148f471fd"
server_key_iv: "8e8e1500e7dc48b641ed5aa515a6449c"
server_key_version: "encryptionkey@0"
I20260808 06:08:18.749114 27908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:08:18.760550 27908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:08:18.763568 27908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:08:18.763742 27908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:08:18.764315 27908 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:08:18.765442 28020 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 06:08:18.769572 27908 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 06:08:18.769627 27908 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260808 06:08:18.769676 27908 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 06:08:18.771330 27908 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 06:08:18.771389 27908 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260808 06:08:18.771679 28020 tablet_bootstrap.cc:492] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd: Bootstrap starting.
I20260808 06:08:18.781708 27908 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.129:33379
I20260808 06:08:18.781988 28201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.129:33379 every 8 connection(s)
I20260808 06:08:18.782934 27908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
I20260808 06:08:18.785336 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 27908
I20260808 06:08:18.785825 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.130:45151
--local_ip_for_outbound_sockets=127.1.151.130
--tserver_master_addrs=127.1.151.190:39415
--webserver_port=36177
--webserver_interface=127.1.151.130
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:35001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 06:08:18.794068 28020 log.cc:824] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd: Log is configured to *not* fsync() on all Append() calls
I20260808 06:08:18.803210 28020 tablet_bootstrap.cc:492] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 06:08:18.803615 28020 tablet_bootstrap.cc:492] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd: Bootstrap complete.
I20260808 06:08:18.805086 28020 ts_tablet_manager.cc:1397] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd: Time spent bootstrapping tablet: real 0.034s user 0.019s sys 0.011s
I20260808 06:08:18.808329 28020 raft_consensus.cc:348] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e76571d8944ab7a9226db3a83a8719" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 46043 } } peers { permanent_uuid: "05cb149b8a664d0b89186253ff773f8b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45151 } } peers { permanent_uuid: "d0e15846c2fc43a29763dee9c94558dd" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 33379 } }
I20260808 06:08:18.808862 28020 raft_consensus.cc:729] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0e15846c2fc43a29763dee9c94558dd, State: Initialized, Role: FOLLOWER
I20260808 06:08:18.809366 28020 consensus_queue.cc:260] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e76571d8944ab7a9226db3a83a8719" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 46043 } } peers { permanent_uuid: "05cb149b8a664d0b89186253ff773f8b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45151 } } peers { permanent_uuid: "d0e15846c2fc43a29763dee9c94558dd" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 33379 } }
I20260808 06:08:18.810452 28020 ts_tablet_manager.cc:1428] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd: Time spent starting tablet: real 0.005s user 0.007s sys 0.001s
W20260808 06:08:18.846589 28204 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:08:18.785220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.129:55895)
0808 06:08:18.786488 (+ 1268us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:18.786557 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:18.787646 (+ 1089us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:18.845820 (+ 58174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1004,"thread_start_us":140,"threads_started":1}
W20260808 06:08:18.847183 28202 heartbeater.cc:644] Failed to heartbeat to 127.1.151.190:39415 (0 consecutive failures): Network error: Failed to ping master at 127.1.151.190:39415: Client connection negotiation failed: client connection to 127.1.151.190:39415: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 06:08:18.847929 28204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:18.847676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.129:54713)
0808 06:08:18.847805 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:18.847871 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 06:08:18.848582 28204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:18.848399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.129:37739)
0808 06:08:18.848492 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:18.848533 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 06:08:18.848744 28202 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 06:08:18.904664 28205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:18.904903 28205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:18.904946 28205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:18.908903 28205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:18.908955 28205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:18.909024 28205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.130
I20260808 06:08:18.914049 28205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:35001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.130:45151
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.151.130
--webserver_port=36177
--tserver_master_addrs=127.1.151.190:39415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.130
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:18.914980 28205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:18.915889 28205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:08:18.925079 28215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:18.925056 28218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:18.925189 28205 server_base.cc:1034] running on GCE node
W20260808 06:08:18.925060 28221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:18.925661 28205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:18.926319 28205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:18.927471 28205 hybrid_clock.cc:648] HybridClock initialized: now 1786169298927441 us; error 41 us; skew 500 ppm
Aug 08 06:08:18 dist-test-slave-zj8q krb5kdc[25996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169298, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:08:18.930541 28205 init.cc:377] Logged in from keytab as kudu/127.1.151.130@KRBTEST.COM (short username kudu)
I20260808 06:08:18.931396 28205 webserver.cc:460] Webserver started at http://127.1.151.130:36177/ using document root <none> and password file <none>
I20260808 06:08:18.931955 28205 fs_manager.cc:359] Metadata directory not provided
I20260808 06:08:18.932045 28205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:08:18.935935 28205 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 06:08:18.937906 28233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:18.939150 28205 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:08:18.939277 28205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/wal
uuid: "05cb149b8a664d0b89186253ff773f8b"
format_stamp: "Formatted at 2026-08-08 06:07:36 on dist-test-slave-zj8q"
server_key: "7e3281fcb12fe62372dd49fdb7451ea4"
server_key_iv: "4614d4e9c7bab6ef96db0e46cc08cd4c"
server_key_version: "encryptionkey@0"
I20260808 06:08:18.939800 28205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:08:18.961526 28205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:08:18.964327 28205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:08:18.964504 28205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:08:18.965087 28205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:08:18.965927 28205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:08:18.965979 28205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:18.966041 28205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:08:18.966068 28205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:18.974738 28205 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.130:45151
I20260808 06:08:18.974787 28429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.130:45151 every 8 connection(s)
I20260808 06:08:18.975693 28205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
I20260808 06:08:18.979321 28432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:18.977672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.130:38447)
0808 06:08:18.978182 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:18.978628 (+ 446us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":109,"threads_started":1}
W20260808 06:08:18.979758 28430 heartbeater.cc:644] Failed to heartbeat to 127.1.151.190:39415 (0 consecutive failures): Network error: Failed to ping master at 127.1.151.190:39415: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
I20260808 06:08:18.980363 28432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:18.980107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.130:51833)
0808 06:08:18.980255 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:18.980306 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260808 06:08:18.980988 28432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:18.980795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.130:39167)
0808 06:08:18.980892 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:18.980936 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 06:08:18.981178 28430 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 06:08:18.984975 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 28205
I20260808 06:08:18.985363 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.131:46043
--local_ip_for_outbound_sockets=127.1.151.131
--tserver_master_addrs=127.1.151.190:39415
--webserver_port=38245
--webserver_interface=127.1.151.131
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:35001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:08:19.094787 28434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:19.095027 28434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:19.095068 28434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:19.099059 28434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:19.099109 28434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:19.099179 28434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.131
I20260808 06:08:19.104362 28434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:35001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.131:46043
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.151.131
--webserver_port=38245
--tserver_master_addrs=127.1.151.190:39415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.131
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:19.105333 28434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:19.106364 28434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:08:19.115552 28440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:19.115577 28439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:19.115577 28443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:19.115782 28434 server_base.cc:1034] running on GCE node
I20260808 06:08:19.116154 28434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:19.116758 28434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:19.117928 28434 hybrid_clock.cc:648] HybridClock initialized: now 1786169299117896 us; error 43 us; skew 500 ppm
Aug 08 06:08:19 dist-test-slave-zj8q krb5kdc[25996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169299, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:08:19.120682 28434 init.cc:377] Logged in from keytab as kudu/127.1.151.131@KRBTEST.COM (short username kudu)
I20260808 06:08:19.121687 28434 webserver.cc:460] Webserver started at http://127.1.151.131:38245/ using document root <none> and password file <none>
I20260808 06:08:19.122252 28434 fs_manager.cc:359] Metadata directory not provided
I20260808 06:08:19.122318 28434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:08:19.126010 28434 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:08:19.128078 28456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:19.129273 28434 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:08:19.129377 28434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/wal
uuid: "e9e76571d8944ab7a9226db3a83a8719"
format_stamp: "Formatted at 2026-08-08 06:07:36 on dist-test-slave-zj8q"
server_key: "54b70e21e3349c4ac35822ad6ed91275"
server_key_iv: "fe06e4a73ea64348933d25d7624ccf39"
server_key_version: "encryptionkey@0"
I20260808 06:08:19.129904 28434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:08:19.148048 28434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:08:19.150837 28434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:08:19.151010 28434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:08:19.151616 28434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:08:19.152467 28434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:08:19.152535 28434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:19.152594 28434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:08:19.152626 28434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:19.161233 28434 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.131:46043
I20260808 06:08:19.161268 28648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.131:46043 every 8 connection(s)
I20260808 06:08:19.162319 28434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
I20260808 06:08:19.166076 28651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:19.164434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.131:51651)
0808 06:08:19.164953 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:19.165387 (+ 434us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":119,"threads_started":1}
W20260808 06:08:19.166517 28649 heartbeater.cc:644] Failed to heartbeat to 127.1.151.190:39415 (0 consecutive failures): Network error: Failed to ping master at 127.1.151.190:39415: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
I20260808 06:08:19.167035 28651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:19.166839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.131:51425)
0808 06:08:19.166935 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:19.166980 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 06:08:19.167595 28651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:19.167370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.131:38645)
0808 06:08:19.167499 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:19.167544 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260808 06:08:19.167829 28649 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 06:08:19.172092 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 28434
I20260808 06:08:19.849749 28655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:19.849288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.129:35271)
0808 06:08:19.849541 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:19.849594 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":77,"threads_started":1}
I20260808 06:08:19.982074 28657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:19.981692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.130:43301)
0808 06:08:19.981922 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:19.981973 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":83,"threads_started":1}
I20260808 06:08:20.168723 28660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:20.168337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.131:54373)
0808 06:08:20.168565 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:20.168621 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":69,"threads_started":1}
I20260808 06:08:20.344957 28661 raft_consensus.cc:482] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:08:20.345155 28661 raft_consensus.cc:504] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e76571d8944ab7a9226db3a83a8719" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 46043 } } peers { permanent_uuid: "05cb149b8a664d0b89186253ff773f8b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45151 } } peers { permanent_uuid: "d0e15846c2fc43a29763dee9c94558dd" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 33379 } }
I20260808 06:08:20.346330 28661 leader_election.cc:288] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9e76571d8944ab7a9226db3a83a8719 (127.1.151.131:46043), 05cb149b8a664d0b89186253ff773f8b (127.1.151.130:45151)
Aug 08 06:08:20 dist-test-slave-zj8q krb5kdc[25996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169298, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for kudu/127.1.151.131@KRBTEST.COM
Aug 08 06:08:20 dist-test-slave-zj8q krb5kdc[25996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169298, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for kudu/127.1.151.130@KRBTEST.COM
I20260808 06:08:20.359754 28662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:20.346596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:45151 (local address 127.1.151.129:39601)
0808 06:08:20.346946 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:20.346960 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:20.347079 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:20.348519 (+ 1440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:20.348525 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:20.349091 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:08:20.349540 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:20.349555 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:20.351145 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:20.351148 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:20.351901 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:20.351911 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:20.352642 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:20.352646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:20.352835 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:20.353432 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:08:20.353454 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:08:20.355502 (+ 2048us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:08:20.357379 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:08:20.357382 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:08:20.357383 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:08:20.357718 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:08:20.358043 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:08:20.358046 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:08:20.358048 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:08:20.358240 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:08:20.358825 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:08:20.358831 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:08:20.359057 (+ 226us) client_negotiation.cc:769] Sending connection context
0808 06:08:20.359218 (+ 161us) client_negotiation.cc:240] Negotiation successful
0808 06:08:20.359550 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":112,"spinlock_wait_cycles":19328,"thread_start_us":73,"threads_started":1}
I20260808 06:08:20.359764 28663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:20.346696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:51409 (local address 127.1.151.131:46043)
0808 06:08:20.347047 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:20.347072 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:20.347121 (+ 49us) server_negotiation.cc:407] Connection header received
0808 06:08:20.347810 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:20.347816 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:20.348153 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:20.348403 (+ 250us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:20.349713 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:20.350823 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:20.352029 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:20.352519 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:20.352674 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:20.355437 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:20.355444 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:20.355459 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:08:20.355491 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:20.357206 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:20.357829 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:20.357834 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:20.357838 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:20.357950 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:20.358341 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:20.358343 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:20.358345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:20.358707 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:20.358825 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:20.359353 (+ 528us) server_negotiation.cc:299] Negotiation successful
0808 06:08:20.359600 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":80,"threads_started":1}
I20260808 06:08:20.359803 28664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:20.346839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:39601 (local address 127.1.151.130:45151)
0808 06:08:20.347188 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:20.347212 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:20.347246 (+ 34us) server_negotiation.cc:407] Connection header received
0808 06:08:20.347901 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:20.347907 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:20.348207 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:20.348485 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:20.349861 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:20.350999 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:20.352012 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:20.352517 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:20.352685 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:20.355700 (+ 3015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:20.355705 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:20.355716 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:08:20.355744 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:20.357257 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:20.357828 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:20.357833 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:20.357837 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:20.357951 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:20.358330 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:20.358334 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:20.358335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:20.358710 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:20.358828 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:20.359355 (+ 527us) server_negotiation.cc:299] Negotiation successful
0808 06:08:20.359615 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":128,"threads_started":1}
I20260808 06:08:20.359790 28655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:20.346499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:46043 (local address 127.1.151.129:51409)
0808 06:08:20.346612 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:20.346628 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:20.346771 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:20.348518 (+ 1747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:20.348524 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:20.349093 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:08:20.349540 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:20.349555 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:20.350963 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:20.350967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:20.351910 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:20.351922 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:20.352634 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:20.352638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:20.352832 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:20.353472 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:08:20.353484 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:08:20.355213 (+ 1729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:08:20.357331 (+ 2118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:08:20.357338 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:08:20.357351 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:08:20.357720 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:08:20.358058 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:08:20.358062 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:08:20.358063 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:08:20.358248 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:08:20.358823 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:08:20.358829 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:08:20.359066 (+ 237us) client_negotiation.cc:769] Sending connection context
0808 06:08:20.359217 (+ 151us) client_negotiation.cc:240] Negotiation successful
0808 06:08:20.359626 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":215,"spinlock_wait_cycles":36352}
W20260808 06:08:20.362498 28016 leader_election.cc:341] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 05cb149b8a664d0b89186253ff773f8b (127.1.151.130:45151): Not found: Tablet not found: ee92ac18a1b24b11afdbed8e566767ae
W20260808 06:08:20.362636 28011 leader_election.cc:341] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e9e76571d8944ab7a9226db3a83a8719 (127.1.151.131:46043): Not found: Tablet not found: ee92ac18a1b24b11afdbed8e566767ae
I20260808 06:08:20.362730 28011 leader_election.cc:302] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d0e15846c2fc43a29763dee9c94558dd; no voters: 05cb149b8a664d0b89186253ff773f8b, e9e76571d8944ab7a9226db3a83a8719
I20260808 06:08:20.362965 28661 raft_consensus.cc:2726] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 06:08:20.850750 28655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:20.850489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.129:41413)
0808 06:08:20.850606 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:20.850661 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 06:08:20.983114 28665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:20.982735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.130:33375)
0808 06:08:20.982974 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:20.983027 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":75,"threads_started":1}
I20260808 06:08:21.169821 28666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:21.169426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.131:34407)
0808 06:08:21.169673 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:21.169730 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":73,"threads_started":1}
I20260808 06:08:21.851899 28669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:21.851479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.129:39557)
0808 06:08:21.851742 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:21.851799 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":78,"threads_started":1}
I20260808 06:08:21.984211 28670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:21.983772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.130:53159)
0808 06:08:21.984032 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:21.984087 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":74,"threads_started":1}
I20260808 06:08:22.170957 28672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:22.170517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.131:43535)
0808 06:08:22.170790 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:22.170847 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":73,"threads_started":1}
I20260808 06:08:22.533012 28673 raft_consensus.cc:482] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:08:22.533185 28673 raft_consensus.cc:504] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e76571d8944ab7a9226db3a83a8719" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 46043 } } peers { permanent_uuid: "05cb149b8a664d0b89186253ff773f8b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45151 } } peers { permanent_uuid: "d0e15846c2fc43a29763dee9c94558dd" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 33379 } }
I20260808 06:08:22.533656 28673 leader_election.cc:288] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9e76571d8944ab7a9226db3a83a8719 (127.1.151.131:46043), 05cb149b8a664d0b89186253ff773f8b (127.1.151.130:45151)
W20260808 06:08:22.534471 28016 leader_election.cc:341] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 05cb149b8a664d0b89186253ff773f8b (127.1.151.130:45151): Not found: Tablet not found: ee92ac18a1b24b11afdbed8e566767ae
W20260808 06:08:22.534634 28011 leader_election.cc:341] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e9e76571d8944ab7a9226db3a83a8719 (127.1.151.131:46043): Not found: Tablet not found: ee92ac18a1b24b11afdbed8e566767ae
I20260808 06:08:22.534722 28011 leader_election.cc:302] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d0e15846c2fc43a29763dee9c94558dd; no voters: 05cb149b8a664d0b89186253ff773f8b, e9e76571d8944ab7a9226db3a83a8719
I20260808 06:08:22.534873 28673 raft_consensus.cc:2726] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 06:08:22.853091 28676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:22.852669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.129:36331)
0808 06:08:22.852929 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:22.852988 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":80,"threads_started":1}
I20260808 06:08:22.985311 28679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:22.984913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.130:56263)
0808 06:08:22.985165 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:22.985221 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":71,"threads_started":1}
I20260808 06:08:23.172051 28680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:23.171647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.131:35167)
0808 06:08:23.171903 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:23.171958 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":74,"threads_started":1}
I20260808 06:08:23.854630 28682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:23.854074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.129:55677)
0808 06:08:23.854458 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:23.854523 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":132,"threads_started":1}
I20260808 06:08:23.986544 28683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:23.986128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.130:46309)
0808 06:08:23.986401 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:23.986454 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":77,"threads_started":1}
I20260808 06:08:24.173286 28685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:24.172812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.131:60843)
0808 06:08:24.173108 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:24.173184 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":99,"threads_started":1}
I20260808 06:08:24.855816 28687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:24.855387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.129:52287)
0808 06:08:24.855656 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:24.855714 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":97,"threads_started":1}
I20260808 06:08:24.987627 28689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:24.987256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.130:41931)
0808 06:08:24.987484 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:24.987537 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":69,"threads_started":1}
I20260808 06:08:25.174458 28690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:25.174005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.131:51349)
0808 06:08:25.174307 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:25.174362 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":81,"threads_started":1}
I20260808 06:08:25.339564 28692 raft_consensus.cc:482] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:08:25.339705 28692 raft_consensus.cc:504] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e76571d8944ab7a9226db3a83a8719" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 46043 } } peers { permanent_uuid: "05cb149b8a664d0b89186253ff773f8b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45151 } } peers { permanent_uuid: "d0e15846c2fc43a29763dee9c94558dd" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 33379 } }
I20260808 06:08:25.340117 28692 leader_election.cc:288] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9e76571d8944ab7a9226db3a83a8719 (127.1.151.131:46043), 05cb149b8a664d0b89186253ff773f8b (127.1.151.130:45151)
W20260808 06:08:25.340973 28011 leader_election.cc:341] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e9e76571d8944ab7a9226db3a83a8719 (127.1.151.131:46043): Not found: Tablet not found: ee92ac18a1b24b11afdbed8e566767ae
W20260808 06:08:25.341104 28016 leader_election.cc:341] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 05cb149b8a664d0b89186253ff773f8b (127.1.151.130:45151): Not found: Tablet not found: ee92ac18a1b24b11afdbed8e566767ae
I20260808 06:08:25.341203 28016 leader_election.cc:302] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d0e15846c2fc43a29763dee9c94558dd; no voters: 05cb149b8a664d0b89186253ff773f8b, e9e76571d8944ab7a9226db3a83a8719
I20260808 06:08:25.341375 28692 raft_consensus.cc:2726] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 06:08:25.856972 28694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:25.856547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.129:34241)
0808 06:08:25.856817 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:25.856872 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":106,"threads_started":1}
I20260808 06:08:25.988847 28697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:25.988336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.130:41725)
0808 06:08:25.988696 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:25.988753 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":74,"threads_started":1}
I20260808 06:08:26.175642 28698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:26.175184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.131:43129)
0808 06:08:26.175483 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:26.175540 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":88,"threads_started":1}
I20260808 06:08:26.858114 28699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:26.857668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.129:43825)
0808 06:08:26.857947 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:26.858005 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":95,"threads_started":1}
I20260808 06:08:26.942411 28700 raft_consensus.cc:482] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:08:26.942538 28700 raft_consensus.cc:504] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e76571d8944ab7a9226db3a83a8719" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 46043 } } peers { permanent_uuid: "05cb149b8a664d0b89186253ff773f8b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45151 } } peers { permanent_uuid: "d0e15846c2fc43a29763dee9c94558dd" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 33379 } }
I20260808 06:08:26.943158 28700 leader_election.cc:288] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9e76571d8944ab7a9226db3a83a8719 (127.1.151.131:46043), 05cb149b8a664d0b89186253ff773f8b (127.1.151.130:45151)
W20260808 06:08:26.943949 28011 leader_election.cc:341] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e9e76571d8944ab7a9226db3a83a8719 (127.1.151.131:46043): Not found: Tablet not found: ee92ac18a1b24b11afdbed8e566767ae
W20260808 06:08:26.944073 28016 leader_election.cc:341] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 05cb149b8a664d0b89186253ff773f8b (127.1.151.130:45151): Not found: Tablet not found: ee92ac18a1b24b11afdbed8e566767ae
I20260808 06:08:26.944141 28016 leader_election.cc:302] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d0e15846c2fc43a29763dee9c94558dd; no voters: 05cb149b8a664d0b89186253ff773f8b, e9e76571d8944ab7a9226db3a83a8719
I20260808 06:08:26.944276 28700 raft_consensus.cc:2726] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 06:08:26.989940 28702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:26.989568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.130:58823)
0808 06:08:26.989802 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:26.989854 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":75,"threads_started":1}
I20260808 06:08:27.176710 28706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:27.176300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.131:37137)
0808 06:08:27.176537 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:27.176589 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":72,"threads_started":1}
I20260808 06:08:27.859251 28707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:27.858825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.129:39875)
0808 06:08:27.859091 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:27.859151 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":78,"threads_started":1}
I20260808 06:08:27.990965 28708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:27.990577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.130:57579)
0808 06:08:27.990815 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:27.990871 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":79,"threads_started":1}
I20260808 06:08:28.177819 28709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:28.177416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.131:41533)
0808 06:08:28.177671 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:28.177724 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":90,"threads_started":1}
I20260808 06:08:28.860463 28710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:28.860029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.129:39967)
0808 06:08:28.860304 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:28.860362 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":93,"threads_started":1}
I20260808 06:08:28.992060 28711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:28.991685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.130:41699)
0808 06:08:28.991918 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:28.991970 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":69,"threads_started":1}
I20260808 06:08:29.178966 28715 raft_consensus.cc:482] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:08:29.179052 28713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:29.178570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.131:39507)
0808 06:08:29.178889 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:29.178954 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":76,"threads_started":1}
I20260808 06:08:29.179118 28715 raft_consensus.cc:504] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e76571d8944ab7a9226db3a83a8719" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 46043 } } peers { permanent_uuid: "05cb149b8a664d0b89186253ff773f8b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45151 } } peers { permanent_uuid: "d0e15846c2fc43a29763dee9c94558dd" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 33379 } }
I20260808 06:08:29.179586 28715 leader_election.cc:288] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9e76571d8944ab7a9226db3a83a8719 (127.1.151.131:46043), 05cb149b8a664d0b89186253ff773f8b (127.1.151.130:45151)
W20260808 06:08:29.180229 28011 leader_election.cc:341] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e9e76571d8944ab7a9226db3a83a8719 (127.1.151.131:46043): Not found: Tablet not found: ee92ac18a1b24b11afdbed8e566767ae
W20260808 06:08:29.180318 28016 leader_election.cc:341] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 05cb149b8a664d0b89186253ff773f8b (127.1.151.130:45151): Not found: Tablet not found: ee92ac18a1b24b11afdbed8e566767ae
I20260808 06:08:29.180423 28016 leader_election.cc:302] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d0e15846c2fc43a29763dee9c94558dd; no voters: 05cb149b8a664d0b89186253ff773f8b, e9e76571d8944ab7a9226db3a83a8719
I20260808 06:08:29.180572 28715 raft_consensus.cc:2726] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 06:08:29.861609 28716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:29.861127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.129:37959)
0808 06:08:29.861440 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:29.861497 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":83,"threads_started":1}
I20260808 06:08:29.993167 28717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:29.992757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.130:33973)
0808 06:08:29.993007 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:29.993064 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":72,"threads_started":1}
I20260808 06:08:30.180174 28719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:30.179774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.131:41935)
0808 06:08:30.180015 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:30.180067 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":76,"threads_started":1}
I20260808 06:08:30.862754 28720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:30.862316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.129:37333)
0808 06:08:30.862591 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:30.862649 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":85,"threads_started":1}
I20260808 06:08:30.994302 28722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:30.993854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.130:60727)
0808 06:08:30.994095 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:30.994202 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":69,"threads_started":1}
I20260808 06:08:31.181349 28723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:31.180906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.131:39015)
0808 06:08:31.181194 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:31.181253 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":107,"threads_started":1}
I20260808 06:08:31.863931 28724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:31.863463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.129:49121)
0808 06:08:31.863768 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:31.863826 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":92,"threads_started":1}
I20260808 06:08:31.995410 28726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:31.994978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.130:47883)
0808 06:08:31.995250 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:31.995306 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":77,"threads_started":1}
I20260808 06:08:32.182498 28727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:32.182043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.131:54941)
0808 06:08:32.182341 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:32.182399 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":105,"threads_started":1}
I20260808 06:08:32.865063 28728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:32.864630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.129:43323)
0808 06:08:32.864902 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:32.864961 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":77,"threads_started":1}
I20260808 06:08:32.996603 28729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:32.996158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.130:51461)
0808 06:08:32.996439 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:32.996505 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":95,"threads_started":1}
I20260808 06:08:33.183596 28731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:33.183194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.131:35343)
0808 06:08:33.183442 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:33.183497 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":101,"threads_started":1}
I20260808 06:08:33.441052 28733 raft_consensus.cc:482] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:08:33.441190 28733 raft_consensus.cc:504] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e76571d8944ab7a9226db3a83a8719" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 46043 } } peers { permanent_uuid: "05cb149b8a664d0b89186253ff773f8b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45151 } } peers { permanent_uuid: "d0e15846c2fc43a29763dee9c94558dd" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 33379 } }
I20260808 06:08:33.441592 28733 leader_election.cc:288] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9e76571d8944ab7a9226db3a83a8719 (127.1.151.131:46043), 05cb149b8a664d0b89186253ff773f8b (127.1.151.130:45151)
W20260808 06:08:33.442415 28016 leader_election.cc:341] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 05cb149b8a664d0b89186253ff773f8b (127.1.151.130:45151): Not found: Tablet not found: ee92ac18a1b24b11afdbed8e566767ae
W20260808 06:08:33.442528 28011 leader_election.cc:341] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e9e76571d8944ab7a9226db3a83a8719 (127.1.151.131:46043): Not found: Tablet not found: ee92ac18a1b24b11afdbed8e566767ae
I20260808 06:08:33.442625 28011 leader_election.cc:302] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d0e15846c2fc43a29763dee9c94558dd; no voters: 05cb149b8a664d0b89186253ff773f8b, e9e76571d8944ab7a9226db3a83a8719
I20260808 06:08:33.442773 28733 raft_consensus.cc:2726] T ee92ac18a1b24b11afdbed8e566767ae P d0e15846c2fc43a29763dee9c94558dd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 06:08:33.866353 28734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:33.865947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.129:54809)
0808 06:08:33.866201 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:33.866252 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":86,"threads_started":1}
I20260808 06:08:33.997784 28735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:33.997398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.130:41107)
0808 06:08:33.997641 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:33.997693 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":73,"threads_started":1}
I20260808 06:08:34.183279 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 27908
I20260808 06:08:34.188839 28736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:34.186931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39415 (local address 127.1.151.131:33319)
0808 06:08:34.188627 (+ 1696us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:34.188712 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.190:39415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1580,"thread_start_us":1457,"threads_started":1}
I20260808 06:08:34.195886 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 28205
I20260808 06:08:34.215487 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 28434
2026-08-08T06:08:34Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59862 ms)
[----------] 30 tests from SecurityITest (243266 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:08:34 dist-test-slave-zj8q krb5kdc[28740](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:08:34 dist-test-slave-zj8q krb5kdc[28740](info): set up 2 sockets
Aug 08 06:08:34 dist-test-slave-zj8q krb5kdc[28740](info): commencing operation
krb5kdc: starting...
W20260808 06:08:35.384361 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:08:35 dist-test-slave-zj8q krb5kdc[28740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:08:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:08:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.151.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
I20260808 06:08:35.503296 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:41383
--webserver_interface=127.1.151.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:33935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:41383
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:08:35.613615 28763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:35.613864 28763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:35.613919 28763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:35.617830 28763 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:08:35.617882 28763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:35.617908 28763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:35.617929 28763 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:08:35.617954 28763 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:08:35.623054 28763 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:33935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:41383
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:41383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:35.624064 28763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:35.625029 28763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:08:35.633495 28769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:35.633524 28770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:35.633584 28763 server_base.cc:1034] running on GCE node
W20260808 06:08:35.633468 28772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:35.634486 28763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:35.635481 28763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:35.636641 28763 hybrid_clock.cc:648] HybridClock initialized: now 1786169315636624 us; error 35 us; skew 500 ppm
Aug 08 06:08:35 dist-test-slave-zj8q krb5kdc[28740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169315, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:08:35.639365 28763 init.cc:377] Logged in from keytab as kudu/127.1.151.190@KRBTEST.COM (short username kudu)
I20260808 06:08:35.640216 28763 webserver.cc:460] Webserver started at http://127.1.151.190:43625/ using document root <none> and password file <none>
I20260808 06:08:35.640727 28763 fs_manager.cc:359] Metadata directory not provided
I20260808 06:08:35.640789 28763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:08:35.640980 28763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:08:35.643002 28763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "67115259c1224700be14e9ca16775b5d"
format_stamp: "Formatted at 2026-08-08 06:08:35 on dist-test-slave-zj8q"
server_key: "0712b372fa3ad653f24985e0f256ae1b"
server_key_iv: "2d3ee2b64e9798277c6a53f32af0d653"
server_key_version: "encryptionkey@0"
I20260808 06:08:35.643585 28763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "67115259c1224700be14e9ca16775b5d"
format_stamp: "Formatted at 2026-08-08 06:08:35 on dist-test-slave-zj8q"
server_key: "0712b372fa3ad653f24985e0f256ae1b"
server_key_iv: "2d3ee2b64e9798277c6a53f32af0d653"
server_key_version: "encryptionkey@0"
I20260808 06:08:35.647183 28763 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 06:08:35.649361 28788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:35.650561 28763 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:08:35.650676 28763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "67115259c1224700be14e9ca16775b5d"
format_stamp: "Formatted at 2026-08-08 06:08:35 on dist-test-slave-zj8q"
server_key: "0712b372fa3ad653f24985e0f256ae1b"
server_key_iv: "2d3ee2b64e9798277c6a53f32af0d653"
server_key_version: "encryptionkey@0"
I20260808 06:08:35.650767 28763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:08:35.695379 28763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:08:35.698526 28763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:08:35.698721 28763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:08:35.705844 28763 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:41383
I20260808 06:08:35.705879 28878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:41383 every 8 connection(s)
I20260808 06:08:35.706825 28763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:08:35.709522 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 28763
I20260808 06:08:35.709623 28880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:08:35.709664 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:08:35.709988 1630 external_mini_cluster.cc:1428] Setting key 2d389958d010fc79d863afcad87c8431
I20260808 06:08:35.716185 28880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67115259c1224700be14e9ca16775b5d: Bootstrap starting.
I20260808 06:08:35.718827 28880 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67115259c1224700be14e9ca16775b5d: Neither blocks nor log segments found. Creating new log.
Aug 08 06:08:35 dist-test-slave-zj8q krb5kdc[28740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:08:35.719527 28880 log.cc:824] T 00000000000000000000000000000000 P 67115259c1224700be14e9ca16775b5d: Log is configured to *not* fsync() on all Append() calls
I20260808 06:08:35.721279 28880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67115259c1224700be14e9ca16775b5d: No bootstrap required, opened a new log
I20260808 06:08:35.723882 28883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:35.711363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57336 (local address 127.1.151.190:41383)
0808 06:08:35.711772 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:35.711788 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:35.711822 (+ 34us) server_negotiation.cc:407] Connection header received
0808 06:08:35.712719 (+ 897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:35.712738 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:35.713050 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:35.713324 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:35.714289 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:35.715266 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:35.716405 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:35.716881 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:35.716998 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:35.719710 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:35.719716 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:35.719729 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:08:35.719757 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:35.721260 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:35.721680 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:35.721685 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:35.721690 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:35.721802 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:35.722061 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:35.722064 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:35.722065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:35.722469 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:35.722673 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:35.722930 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 06:08:35.723205 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":116,"threads_started":1}
I20260808 06:08:35.724726 28880 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67115259c1224700be14e9ca16775b5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67115259c1224700be14e9ca16775b5d" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 41383 } }
I20260808 06:08:35.724937 28880 raft_consensus.cc:374] T 00000000000000000000000000000000 P 67115259c1224700be14e9ca16775b5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:08:35.724995 28880 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67115259c1224700be14e9ca16775b5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67115259c1224700be14e9ca16775b5d, State: Initialized, Role: FOLLOWER
I20260808 06:08:35.725373 28880 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67115259c1224700be14e9ca16775b5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67115259c1224700be14e9ca16775b5d" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 41383 } }
I20260808 06:08:35.725500 28880 raft_consensus.cc:388] T 00000000000000000000000000000000 P 67115259c1224700be14e9ca16775b5d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:08:35.725576 28880 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67115259c1224700be14e9ca16775b5d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:08:35.725678 28880 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67115259c1224700be14e9ca16775b5d [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:08:35.726583 28880 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67115259c1224700be14e9ca16775b5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67115259c1224700be14e9ca16775b5d" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 41383 } }
I20260808 06:08:35.726894 28880 leader_election.cc:302] T 00000000000000000000000000000000 P 67115259c1224700be14e9ca16775b5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 67115259c1224700be14e9ca16775b5d; no voters:
I20260808 06:08:35.727171 28880 leader_election.cc:288] T 00000000000000000000000000000000 P 67115259c1224700be14e9ca16775b5d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:08:35.727260 28886 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67115259c1224700be14e9ca16775b5d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:08:35.727463 28886 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67115259c1224700be14e9ca16775b5d [term 1 LEADER]: Becoming Leader. State: Replica: 67115259c1224700be14e9ca16775b5d, State: Running, Role: LEADER
I20260808 06:08:35.727782 28886 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67115259c1224700be14e9ca16775b5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67115259c1224700be14e9ca16775b5d" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 41383 } }
I20260808 06:08:35.728296 28880 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67115259c1224700be14e9ca16775b5d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:08:35.729385 28887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67115259c1224700be14e9ca16775b5d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "67115259c1224700be14e9ca16775b5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67115259c1224700be14e9ca16775b5d" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 41383 } } }
I20260808 06:08:35.729421 28888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67115259c1224700be14e9ca16775b5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67115259c1224700be14e9ca16775b5d. Latest consensus state: current_term: 1 leader_uuid: "67115259c1224700be14e9ca16775b5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67115259c1224700be14e9ca16775b5d" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 41383 } } }
I20260808 06:08:35.729509 28887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67115259c1224700be14e9ca16775b5d [sys.catalog]: This master's current role is: LEADER
I20260808 06:08:35.729524 28888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67115259c1224700be14e9ca16775b5d [sys.catalog]: This master's current role is: LEADER
I20260808 06:08:35.730023 28895 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:08:35.732621 28895 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:08:35.737901 28895 catalog_manager.cc:1347] Generated new cluster ID: 59922a7a3912445995f2e9e8c8fa1a7b
I20260808 06:08:35.737967 28895 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:08:35.743634 28895 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:08:35.744518 28895 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:08:35.750831 28895 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 67115259c1224700be14e9ca16775b5d: Generated new TSK 0
I20260808 06:08:35.751519 28895 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.151.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab.
Entry for principal kudu/127.1.151.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab.
I20260808 06:08:35.784310 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.129:0
--local_ip_for_outbound_sockets=127.1.151.129
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:41383
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:33935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:08:35.895376 28915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:35.895612 28915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:35.895653 28915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:35.899726 28915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:35.899778 28915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:35.899845 28915 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.129
I20260808 06:08:35.904883 28915 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:33935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:41383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28915
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.129
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:35.905855 28915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:35.906862 28915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:08:35.916077 28923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:35.916107 28925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:35.916119 28920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:35.916384 28915 server_base.cc:1034] running on GCE node
I20260808 06:08:35.916764 28915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:35.917328 28915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:35.918478 28915 hybrid_clock.cc:648] HybridClock initialized: now 1786169315918463 us; error 27 us; skew 500 ppm
Aug 08 06:08:35 dist-test-slave-zj8q krb5kdc[28740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169315, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:08:35.921104 28915 init.cc:377] Logged in from keytab as kudu/127.1.151.129@KRBTEST.COM (short username kudu)
I20260808 06:08:35.921868 28915 webserver.cc:460] Webserver started at http://127.1.151.129:34263/ using document root <none> and password file <none>
I20260808 06:08:35.922432 28915 fs_manager.cc:359] Metadata directory not provided
I20260808 06:08:35.922495 28915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:08:35.922693 28915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:08:35.924640 28915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-0/data/instance:
uuid: "ae9d8d8d731e4f05bbb368fc611af1e6"
format_stamp: "Formatted at 2026-08-08 06:08:35 on dist-test-slave-zj8q"
server_key: "e894196116f066e6cac0dec7d7b8f0af"
server_key_iv: "61498c0397e46e4f89b96f68dbd16415"
server_key_version: "encryptionkey@0"
I20260808 06:08:35.925155 28915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance:
uuid: "ae9d8d8d731e4f05bbb368fc611af1e6"
format_stamp: "Formatted at 2026-08-08 06:08:35 on dist-test-slave-zj8q"
server_key: "e894196116f066e6cac0dec7d7b8f0af"
server_key_iv: "61498c0397e46e4f89b96f68dbd16415"
server_key_version: "encryptionkey@0"
I20260808 06:08:35.928665 28915 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 06:08:35.930936 28934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:35.932053 28915 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:08:35.932149 28915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-0/wal
uuid: "ae9d8d8d731e4f05bbb368fc611af1e6"
format_stamp: "Formatted at 2026-08-08 06:08:35 on dist-test-slave-zj8q"
server_key: "e894196116f066e6cac0dec7d7b8f0af"
server_key_iv: "61498c0397e46e4f89b96f68dbd16415"
server_key_version: "encryptionkey@0"
I20260808 06:08:35.932242 28915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:08:35.947685 28915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:08:35.950806 28915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:08:35.950996 28915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:08:35.951556 28915 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:08:35.952356 28915 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:08:35.952402 28915 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:35.952442 28915 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:08:35.952472 28915 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:35.960989 28915 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.129:46795
I20260808 06:08:35.961014 29112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.129:46795 every 8 connection(s)
I20260808 06:08:35.961817 28915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
I20260808 06:08:35.970252 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 28915
I20260808 06:08:35.970391 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance
I20260808 06:08:35.970717 1630 external_mini_cluster.cc:1428] Setting key c2be334b3cda4ccce0eaf4edfd92da85
Aug 08 06:08:35 dist-test-slave-zj8q krb5kdc[28740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169315, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:08:35.974855 28883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:35.963514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:54037 (local address 127.1.151.190:41383)
0808 06:08:35.963681 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:35.963685 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:35.964496 (+ 811us) server_negotiation.cc:407] Connection header received
0808 06:08:35.965755 (+ 1259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:35.965758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:35.965801 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:35.965875 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:35.967107 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:35.967662 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:35.968800 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:35.969026 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:35.969098 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:35.971590 (+ 2492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:35.971593 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:35.971595 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:08:35.971619 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:35.972911 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:35.973392 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:35.973395 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:35.973396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:35.973472 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:35.973781 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:35.973785 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:35.973786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:35.973959 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:35.974045 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:35.974565 (+ 520us) server_negotiation.cc:299] Negotiation successful
0808 06:08:35.974696 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 06:08:35.975587 29116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:35.963866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:41383 (local address 127.1.151.129:54037)
0808 06:08:35.964346 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:35.964397 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:35.965575 (+ 1178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:35.966099 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:35.966106 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:35.966495 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:08:35.966974 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:35.966988 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:35.967762 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:35.967764 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:35.968698 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:35.968709 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:35.969187 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:35.969190 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:35.969342 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:35.969828 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:08:35.969844 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:08:35.971451 (+ 1607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:08:35.973019 (+ 1568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:08:35.973025 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:08:35.973038 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:08:35.973296 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:08:35.973555 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:08:35.973558 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:08:35.973561 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:08:35.973694 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:08:35.974058 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:08:35.974065 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:08:35.974344 (+ 279us) client_negotiation.cc:769] Sending connection context
0808 06:08:35.974506 (+ 162us) client_negotiation.cc:240] Negotiation successful
0808 06:08:35.974808 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":102,"threads_started":1}
I20260808 06:08:35.976678 29113 heartbeater.cc:347] Connected to a master server at 127.1.151.190:41383
I20260808 06:08:35.976948 29113 heartbeater.cc:464] Registering TS with master...
I20260808 06:08:35.977509 29113 heartbeater.cc:511] Master 127.1.151.190:41383 requested a full tablet report, sending...
I20260808 06:08:35.979223 28816 ts_manager.cc:194] Registered new tserver with Master: ae9d8d8d731e4f05bbb368fc611af1e6 (127.1.151.129:46795)
I20260808 06:08:35.980729 28816 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.129@KRBTEST.COM'} at 127.1.151.129:54037
WARNING: no policy specified for kudu/127.1.151.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab.
Entry for principal kudu/127.1.151.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab.
I20260808 06:08:35.996642 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.130:0
--local_ip_for_outbound_sockets=127.1.151.130
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:41383
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:33935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:08:36.108585 29121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:36.108809 29121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:36.108865 29121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:36.112764 29121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:36.112819 29121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:36.112895 29121 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.130
I20260808 06:08:36.117898 29121 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:33935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:41383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.130
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:36.118818 29121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:36.119743 29121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 06:08:36.128876 29121 server_base.cc:1034] running on GCE node
W20260808 06:08:36.128851 29128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:36.128883 29130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:36.129139 29134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:36.129487 29121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:36.130116 29121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:36.131315 29121 hybrid_clock.cc:648] HybridClock initialized: now 1786169316131298 us; error 30 us; skew 500 ppm
Aug 08 06:08:36 dist-test-slave-zj8q krb5kdc[28740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169316, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:08:36.133921 29121 init.cc:377] Logged in from keytab as kudu/127.1.151.130@KRBTEST.COM (short username kudu)
I20260808 06:08:36.134805 29121 webserver.cc:460] Webserver started at http://127.1.151.130:38847/ using document root <none> and password file <none>
I20260808 06:08:36.135334 29121 fs_manager.cc:359] Metadata directory not provided
I20260808 06:08:36.135394 29121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:08:36.135552 29121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:08:36.137527 29121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-1/data/instance:
uuid: "5bcbfb0f0c194a6caec64d86e9dd07e6"
format_stamp: "Formatted at 2026-08-08 06:08:36 on dist-test-slave-zj8q"
server_key: "4b35cffbbe2496f7c85b0067b06f7f6b"
server_key_iv: "181c92adac3a587cc6d19e58cfaa4530"
server_key_version: "encryptionkey@0"
I20260808 06:08:36.138015 29121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance:
uuid: "5bcbfb0f0c194a6caec64d86e9dd07e6"
format_stamp: "Formatted at 2026-08-08 06:08:36 on dist-test-slave-zj8q"
server_key: "4b35cffbbe2496f7c85b0067b06f7f6b"
server_key_iv: "181c92adac3a587cc6d19e58cfaa4530"
server_key_version: "encryptionkey@0"
I20260808 06:08:36.141458 29121 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 06:08:36.143751 29147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:36.144908 29121 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 06:08:36.145027 29121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-1/wal
uuid: "5bcbfb0f0c194a6caec64d86e9dd07e6"
format_stamp: "Formatted at 2026-08-08 06:08:36 on dist-test-slave-zj8q"
server_key: "4b35cffbbe2496f7c85b0067b06f7f6b"
server_key_iv: "181c92adac3a587cc6d19e58cfaa4530"
server_key_version: "encryptionkey@0"
I20260808 06:08:36.145118 29121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:08:36.167742 29121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:08:36.170523 29121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:08:36.170699 29121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:08:36.171277 29121 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:08:36.172084 29121 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:08:36.172134 29121 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:36.172186 29121 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:08:36.172223 29121 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:36.180723 29121 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.130:43521
I20260808 06:08:36.180754 29324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.130:43521 every 8 connection(s)
I20260808 06:08:36.181564 29121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
I20260808 06:08:36.182380 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 29121
I20260808 06:08:36.182502 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance
I20260808 06:08:36.182773 1630 external_mini_cluster.cc:1428] Setting key 611fe5d1940ebcdde2712a4d9a455541
Aug 08 06:08:36 dist-test-slave-zj8q krb5kdc[28740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169316, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.151.131@KRBTEST.COM; defaulting to no policy
I20260808 06:08:36.194474 28883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:36.183435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:44729 (local address 127.1.151.190:41383)
0808 06:08:36.183605 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:36.183608 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:36.184504 (+ 896us) server_negotiation.cc:407] Connection header received
0808 06:08:36.185390 (+ 886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:36.185393 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:36.185445 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:36.185531 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:36.186715 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:36.187257 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:36.188378 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:36.188607 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:36.188702 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:36.191040 (+ 2338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:36.191043 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:36.191046 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:08:36.191071 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:36.192468 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:36.192966 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:36.192970 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:36.192972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:36.193050 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:36.193358 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:36.193361 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:36.193363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:36.193536 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:36.193630 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:36.194192 (+ 562us) server_negotiation.cc:299] Negotiation successful
0808 06:08:36.194313 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.131@KRBTEST.COM" created.
I20260808 06:08:36.195286 29330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:36.183856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:41383 (local address 127.1.151.130:44729)
0808 06:08:36.184353 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:36.184394 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:36.185185 (+ 791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:36.185708 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:36.185715 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:36.186063 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:08:36.186572 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:36.186586 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:36.187378 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:36.187381 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:36.188256 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:36.188266 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:36.188744 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:36.188748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:36.188909 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:36.189453 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:08:36.189469 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:08:36.190883 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:08:36.192585 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:08:36.192590 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:08:36.192604 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:08:36.192847 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:08:36.193134 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:08:36.193137 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:08:36.193139 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:08:36.193271 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:08:36.193641 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:08:36.193648 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:08:36.193936 (+ 288us) client_negotiation.cc:769] Sending connection context
0808 06:08:36.194108 (+ 172us) client_negotiation.cc:240] Negotiation successful
0808 06:08:36.194429 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":137,"threads_started":1}
I20260808 06:08:36.196389 29325 heartbeater.cc:347] Connected to a master server at 127.1.151.190:41383
I20260808 06:08:36.196689 29325 heartbeater.cc:464] Registering TS with master...
I20260808 06:08:36.197191 29325 heartbeater.cc:511] Master 127.1.151.190:41383 requested a full tablet report, sending...
I20260808 06:08:36.198387 28816 ts_manager.cc:194] Registered new tserver with Master: 5bcbfb0f0c194a6caec64d86e9dd07e6 (127.1.151.130:43521)
I20260808 06:08:36.198982 28816 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.130@KRBTEST.COM'} at 127.1.151.130:44729
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab.
Entry for principal kudu/127.1.151.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab.
I20260808 06:08:36.210264 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.131:0
--local_ip_for_outbound_sockets=127.1.151.131
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:41383
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:33935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:08:36.319659 29333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:36.319892 29333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:36.319947 29333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:36.323913 29333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:36.323968 29333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:36.324045 29333 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.131
I20260808 06:08:36.329511 29333 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:33935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:41383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.131
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:36.330539 29333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:36.331497 29333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:08:36.340631 29343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:36.340658 29333 server_base.cc:1034] running on GCE node
W20260808 06:08:36.340654 29346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:36.340641 29342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:36.341378 29333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:36.341984 29333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:36.343145 29333 hybrid_clock.cc:648] HybridClock initialized: now 1786169316343129 us; error 48 us; skew 500 ppm
Aug 08 06:08:36 dist-test-slave-zj8q krb5kdc[28740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169316, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:08:36.345818 29333 init.cc:377] Logged in from keytab as kudu/127.1.151.131@KRBTEST.COM (short username kudu)
I20260808 06:08:36.346629 29333 webserver.cc:460] Webserver started at http://127.1.151.131:33595/ using document root <none> and password file <none>
I20260808 06:08:36.347127 29333 fs_manager.cc:359] Metadata directory not provided
I20260808 06:08:36.347183 29333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:08:36.347344 29333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:08:36.349376 29333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-2/data/instance:
uuid: "237be1b81b2f4765bb46cec0037d92c3"
format_stamp: "Formatted at 2026-08-08 06:08:36 on dist-test-slave-zj8q"
server_key: "3889546a6bb1bc4c83e564daba50953d"
server_key_iv: "3852a01bf5880fa56c5f28e5198619dd"
server_key_version: "encryptionkey@0"
I20260808 06:08:36.349886 29333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance:
uuid: "237be1b81b2f4765bb46cec0037d92c3"
format_stamp: "Formatted at 2026-08-08 06:08:36 on dist-test-slave-zj8q"
server_key: "3889546a6bb1bc4c83e564daba50953d"
server_key_iv: "3852a01bf5880fa56c5f28e5198619dd"
server_key_version: "encryptionkey@0"
I20260808 06:08:36.353466 29333 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 06:08:36.355867 29360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:36.357023 29333 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 06:08:36.357131 29333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-2/wal
uuid: "237be1b81b2f4765bb46cec0037d92c3"
format_stamp: "Formatted at 2026-08-08 06:08:36 on dist-test-slave-zj8q"
server_key: "3889546a6bb1bc4c83e564daba50953d"
server_key_iv: "3852a01bf5880fa56c5f28e5198619dd"
server_key_version: "encryptionkey@0"
I20260808 06:08:36.357224 29333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:08:36.373399 29333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:08:36.376266 29333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:08:36.376443 29333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:08:36.377048 29333 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:08:36.377895 29333 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:08:36.377942 29333 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:36.377996 29333 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:08:36.378026 29333 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:36.386636 29333 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.131:37135
I20260808 06:08:36.386685 29533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.131:37135 every 8 connection(s)
I20260808 06:08:36.387507 29333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
I20260808 06:08:36.396327 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 29333
Aug 08 06:08:36 dist-test-slave-zj8q krb5kdc[28740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169316, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:08:36.396428 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance
I20260808 06:08:36.396773 1630 external_mini_cluster.cc:1428] Setting key 12a37e40419b9666a9cf4ef0907abf17
I20260808 06:08:36.400629 28883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:36.389378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:41353 (local address 127.1.151.190:41383)
0808 06:08:36.389510 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:36.389514 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:36.390405 (+ 891us) server_negotiation.cc:407] Connection header received
0808 06:08:36.391278 (+ 873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:36.391280 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:36.391327 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:36.391398 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:36.392568 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:36.393104 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:36.394258 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:36.394468 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:36.394543 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:36.397099 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:36.397103 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:36.397106 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:08:36.397136 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:36.398596 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:36.399157 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:36.399162 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:36.399164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:36.399243 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:36.399548 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:36.399551 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:36.399553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:36.399747 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:36.399833 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:36.400336 (+ 503us) server_negotiation.cc:299] Negotiation successful
0808 06:08:36.400457 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 06:08:36.401428 29538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:36.389758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:41383 (local address 127.1.151.131:41353)
0808 06:08:36.390256 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:36.390302 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:36.391082 (+ 780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:36.391552 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:36.391559 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:36.391940 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:08:36.392417 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:36.392432 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:36.393201 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:36.393204 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:36.394127 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:36.394137 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:36.394585 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:36.394587 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:36.394763 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:36.395330 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:08:36.395347 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:08:36.396915 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:08:36.398720 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:08:36.398726 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:08:36.398742 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:08:36.399062 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:08:36.399325 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:08:36.399328 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:08:36.399330 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:08:36.399464 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:08:36.399847 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:08:36.399855 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:08:36.400109 (+ 254us) client_negotiation.cc:769] Sending connection context
0808 06:08:36.400278 (+ 169us) client_negotiation.cc:240] Negotiation successful
0808 06:08:36.400598 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":102,"threads_started":1}
I20260808 06:08:36.402495 29535 heartbeater.cc:347] Connected to a master server at 127.1.151.190:41383
I20260808 06:08:36.402765 29535 heartbeater.cc:464] Registering TS with master...
I20260808 06:08:36.403257 29535 heartbeater.cc:511] Master 127.1.151.190:41383 requested a full tablet report, sending...
I20260808 06:08:36.404285 28816 ts_manager.cc:194] Registered new tserver with Master: 237be1b81b2f4765bb46cec0037d92c3 (127.1.151.131:37135)
I20260808 06:08:36.404841 28816 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.131@KRBTEST.COM'} at 127.1.151.131:41353
I20260808 06:08:36.410666 1630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 06:08:36.419624 28883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:36.412563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57338 (local address 127.1.151.190:41383)
0808 06:08:36.412731 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:36.412734 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:36.412955 (+ 221us) server_negotiation.cc:407] Connection header received
0808 06:08:36.413064 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:36.413066 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:36.413105 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:36.413174 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:36.413897 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:36.414455 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:36.415336 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:36.415548 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:36.415623 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:36.416867 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:36.416870 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:36.416872 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:08:36.416894 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:36.418007 (+ 1113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:36.418444 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:36.418447 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:36.418448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:36.418526 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:36.418808 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:36.418811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:36.418813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:36.419022 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:36.419152 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:36.419370 (+ 218us) server_negotiation.cc:299] Negotiation successful
0808 06:08:36.419478 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260808 06:08:36.422210 28816 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57338:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 06:08:36.425109 28816 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 06:08:36.439213 29563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:36.433167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:46795 (local address 127.0.0.1:36204)
0808 06:08:36.434274 (+ 1107us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:36.434287 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:36.434389 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:36.435143 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:36.435146 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:36.435164 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:08:36.435290 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:36.435298 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:36.436799 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:36.436802 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:36.437686 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:36.437697 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:36.438702 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:36.438706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:36.438812 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:36.438848 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 06:08:36.438954 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 06:08:36.439065 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1051,"mutex_wait_us":6,"thread_start_us":119,"threads_started":1}
I20260808 06:08:36.439419 29562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:36.433214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38158 (local address 127.1.151.131:37135)
0808 06:08:36.434364 (+ 1150us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:36.434378 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:36.434397 (+ 19us) server_negotiation.cc:407] Connection header received
0808 06:08:36.434471 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:36.434477 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:36.434686 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:36.434837 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:08:36.435112 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:36.436279 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:36.438037 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:36.438827 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:36.438963 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:36.439111 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:36.439204 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 06:08:36.439294 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1046,"thread_start_us":174,"threads_started":1}
I20260808 06:08:36.439517 29566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:36.433849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38812 (local address 127.1.151.130:43521)
0808 06:08:36.434624 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:36.434633 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:36.434651 (+ 18us) server_negotiation.cc:407] Connection header received
0808 06:08:36.434712 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:36.434715 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:36.434880 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:36.435020 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:08:36.435481 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:36.436251 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:36.438057 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:36.438639 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:36.438738 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:36.438872 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:36.439286 (+ 414us) server_negotiation.cc:299] Negotiation successful
0808 06:08:36.439386 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":681,"thread_start_us":103,"threads_started":1}
I20260808 06:08:36.439381 29557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:36.433062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:37135 (local address 127.0.0.1:38158)
0808 06:08:36.433858 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:36.433909 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:36.434028 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:36.434839 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:36.434844 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:36.434866 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:08:36.434998 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:36.435005 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:36.436704 (+ 1699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:36.436708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:36.437917 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:36.437929 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:36.438968 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:36.438971 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:36.439065 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:36.439087 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 06:08:36.439175 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 06:08:36.439249 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"mutex_wait_us":28,"thread_start_us":93,"threads_started":1}
I20260808 06:08:36.439875 29564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:36.433849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36204 (local address 127.1.151.129:46795)
0808 06:08:36.434234 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:36.434249 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:36.434388 (+ 139us) server_negotiation.cc:407] Connection header received
0808 06:08:36.434501 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:36.434506 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:36.434679 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:36.434837 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:08:36.435408 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:36.436654 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:36.437812 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:36.438575 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:36.438703 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:36.438850 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:36.439485 (+ 635us) server_negotiation.cc:299] Negotiation successful
0808 06:08:36.439735 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":81,"threads_started":1}
I20260808 06:08:36.439711 29559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:36.433381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:43521 (local address 127.0.0.1:38812)
0808 06:08:36.434352 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:36.434364 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:36.434431 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:36.435179 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:36.435182 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:36.435204 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:08:36.435342 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:36.435349 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:36.436367 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:36.436371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:36.437917 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:36.437929 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:36.438933 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:36.438937 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:36.439029 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:36.439051 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 06:08:36.439122 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 06:08:36.439179 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":923,"mutex_wait_us":73,"spinlock_wait_cycles":896,"thread_start_us":77,"threads_started":1}
I20260808 06:08:36.442947 29015 tablet_service.cc:1467] Processing CreateTablet for tablet e194f49ccf6647b49cd5b467452af897 (DEFAULT_TABLE table=test-table [id=94fc8bc9d28448dfa5134e3dffbb0023]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:08:36.442947 29434 tablet_service.cc:1467] Processing CreateTablet for tablet e194f49ccf6647b49cd5b467452af897 (DEFAULT_TABLE table=test-table [id=94fc8bc9d28448dfa5134e3dffbb0023]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:08:36.444113 29434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e194f49ccf6647b49cd5b467452af897. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:08:36.444108 29015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e194f49ccf6647b49cd5b467452af897. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:08:36.444312 29220 tablet_service.cc:1467] Processing CreateTablet for tablet e194f49ccf6647b49cd5b467452af897 (DEFAULT_TABLE table=test-table [id=94fc8bc9d28448dfa5134e3dffbb0023]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:08:36.445123 29220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e194f49ccf6647b49cd5b467452af897. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:08:36.450809 29567 tablet_bootstrap.cc:492] T e194f49ccf6647b49cd5b467452af897 P 5bcbfb0f0c194a6caec64d86e9dd07e6: Bootstrap starting.
I20260808 06:08:36.451489 29568 tablet_bootstrap.cc:492] T e194f49ccf6647b49cd5b467452af897 P 237be1b81b2f4765bb46cec0037d92c3: Bootstrap starting.
I20260808 06:08:36.452396 29570 tablet_bootstrap.cc:492] T e194f49ccf6647b49cd5b467452af897 P ae9d8d8d731e4f05bbb368fc611af1e6: Bootstrap starting.
I20260808 06:08:36.453814 29568 tablet_bootstrap.cc:654] T e194f49ccf6647b49cd5b467452af897 P 237be1b81b2f4765bb46cec0037d92c3: Neither blocks nor log segments found. Creating new log.
I20260808 06:08:36.454317 29567 tablet_bootstrap.cc:654] T e194f49ccf6647b49cd5b467452af897 P 5bcbfb0f0c194a6caec64d86e9dd07e6: Neither blocks nor log segments found. Creating new log.
I20260808 06:08:36.454427 29570 tablet_bootstrap.cc:654] T e194f49ccf6647b49cd5b467452af897 P ae9d8d8d731e4f05bbb368fc611af1e6: Neither blocks nor log segments found. Creating new log.
I20260808 06:08:36.455077 29567 log.cc:824] T e194f49ccf6647b49cd5b467452af897 P 5bcbfb0f0c194a6caec64d86e9dd07e6: Log is configured to *not* fsync() on all Append() calls
I20260808 06:08:36.455077 29568 log.cc:824] T e194f49ccf6647b49cd5b467452af897 P 237be1b81b2f4765bb46cec0037d92c3: Log is configured to *not* fsync() on all Append() calls
I20260808 06:08:36.455509 29570 log.cc:824] T e194f49ccf6647b49cd5b467452af897 P ae9d8d8d731e4f05bbb368fc611af1e6: Log is configured to *not* fsync() on all Append() calls
I20260808 06:08:36.457090 29568 tablet_bootstrap.cc:492] T e194f49ccf6647b49cd5b467452af897 P 237be1b81b2f4765bb46cec0037d92c3: No bootstrap required, opened a new log
I20260808 06:08:36.457090 29567 tablet_bootstrap.cc:492] T e194f49ccf6647b49cd5b467452af897 P 5bcbfb0f0c194a6caec64d86e9dd07e6: No bootstrap required, opened a new log
I20260808 06:08:36.457154 29570 tablet_bootstrap.cc:492] T e194f49ccf6647b49cd5b467452af897 P ae9d8d8d731e4f05bbb368fc611af1e6: No bootstrap required, opened a new log
I20260808 06:08:36.457285 29568 ts_tablet_manager.cc:1397] T e194f49ccf6647b49cd5b467452af897 P 237be1b81b2f4765bb46cec0037d92c3: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 06:08:36.457285 29567 ts_tablet_manager.cc:1397] T e194f49ccf6647b49cd5b467452af897 P 5bcbfb0f0c194a6caec64d86e9dd07e6: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260808 06:08:36.457348 29570 ts_tablet_manager.cc:1397] T e194f49ccf6647b49cd5b467452af897 P ae9d8d8d731e4f05bbb368fc611af1e6: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260808 06:08:36.460471 29570 raft_consensus.cc:348] T e194f49ccf6647b49cd5b467452af897 P ae9d8d8d731e4f05bbb368fc611af1e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "237be1b81b2f4765bb46cec0037d92c3" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 37135 } } peers { permanent_uuid: "ae9d8d8d731e4f05bbb368fc611af1e6" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 46795 } } peers { permanent_uuid: "5bcbfb0f0c194a6caec64d86e9dd07e6" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 43521 } }
I20260808 06:08:36.460716 29570 raft_consensus.cc:374] T e194f49ccf6647b49cd5b467452af897 P ae9d8d8d731e4f05bbb368fc611af1e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:08:36.460789 29570 raft_consensus.cc:729] T e194f49ccf6647b49cd5b467452af897 P ae9d8d8d731e4f05bbb368fc611af1e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae9d8d8d731e4f05bbb368fc611af1e6, State: Initialized, Role: FOLLOWER
I20260808 06:08:36.460990 29568 raft_consensus.cc:348] T e194f49ccf6647b49cd5b467452af897 P 237be1b81b2f4765bb46cec0037d92c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "237be1b81b2f4765bb46cec0037d92c3" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 37135 } } peers { permanent_uuid: "ae9d8d8d731e4f05bbb368fc611af1e6" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 46795 } } peers { permanent_uuid: "5bcbfb0f0c194a6caec64d86e9dd07e6" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 43521 } }
I20260808 06:08:36.460990 29567 raft_consensus.cc:348] T e194f49ccf6647b49cd5b467452af897 P 5bcbfb0f0c194a6caec64d86e9dd07e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "237be1b81b2f4765bb46cec0037d92c3" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 37135 } } peers { permanent_uuid: "ae9d8d8d731e4f05bbb368fc611af1e6" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 46795 } } peers { permanent_uuid: "5bcbfb0f0c194a6caec64d86e9dd07e6" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 43521 } }
I20260808 06:08:36.461251 29568 raft_consensus.cc:374] T e194f49ccf6647b49cd5b467452af897 P 237be1b81b2f4765bb46cec0037d92c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:08:36.461251 29567 raft_consensus.cc:374] T e194f49ccf6647b49cd5b467452af897 P 5bcbfb0f0c194a6caec64d86e9dd07e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:08:36.461315 29567 raft_consensus.cc:729] T e194f49ccf6647b49cd5b467452af897 P 5bcbfb0f0c194a6caec64d86e9dd07e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bcbfb0f0c194a6caec64d86e9dd07e6, State: Initialized, Role: FOLLOWER
I20260808 06:08:36.461243 29570 consensus_queue.cc:260] T e194f49ccf6647b49cd5b467452af897 P ae9d8d8d731e4f05bbb368fc611af1e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "237be1b81b2f4765bb46cec0037d92c3" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 37135 } } peers { permanent_uuid: "ae9d8d8d731e4f05bbb368fc611af1e6" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 46795 } } peers { permanent_uuid: "5bcbfb0f0c194a6caec64d86e9dd07e6" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 43521 } }
I20260808 06:08:36.461352 29568 raft_consensus.cc:729] T e194f49ccf6647b49cd5b467452af897 P 237be1b81b2f4765bb46cec0037d92c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 237be1b81b2f4765bb46cec0037d92c3, State: Initialized, Role: FOLLOWER
I20260808 06:08:36.461777 29568 consensus_queue.cc:260] T e194f49ccf6647b49cd5b467452af897 P 237be1b81b2f4765bb46cec0037d92c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "237be1b81b2f4765bb46cec0037d92c3" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 37135 } } peers { permanent_uuid: "ae9d8d8d731e4f05bbb368fc611af1e6" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 46795 } } peers { permanent_uuid: "5bcbfb0f0c194a6caec64d86e9dd07e6" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 43521 } }
I20260808 06:08:36.461777 29567 consensus_queue.cc:260] T e194f49ccf6647b49cd5b467452af897 P 5bcbfb0f0c194a6caec64d86e9dd07e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "237be1b81b2f4765bb46cec0037d92c3" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 37135 } } peers { permanent_uuid: "ae9d8d8d731e4f05bbb368fc611af1e6" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 46795 } } peers { permanent_uuid: "5bcbfb0f0c194a6caec64d86e9dd07e6" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 43521 } }
I20260808 06:08:36.462185 29570 ts_tablet_manager.cc:1428] T e194f49ccf6647b49cd5b467452af897 P ae9d8d8d731e4f05bbb368fc611af1e6: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 06:08:36.462463 29325 heartbeater.cc:503] Master 127.1.151.190:41383 was elected leader, sending a full tablet report...
I20260808 06:08:36.462517 29113 heartbeater.cc:503] Master 127.1.151.190:41383 was elected leader, sending a full tablet report...
I20260808 06:08:36.462756 29568 ts_tablet_manager.cc:1428] T e194f49ccf6647b49cd5b467452af897 P 237be1b81b2f4765bb46cec0037d92c3: Time spent starting tablet: real 0.005s user 0.006s sys 0.001s
W20260808 06:08:36.462831 29114 tablet.cc:2369] T e194f49ccf6647b49cd5b467452af897 P ae9d8d8d731e4f05bbb368fc611af1e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 06:08:36.463052 29535 heartbeater.cc:503] Master 127.1.151.190:41383 was elected leader, sending a full tablet report...
I20260808 06:08:36.463827 29567 ts_tablet_manager.cc:1428] T e194f49ccf6647b49cd5b467452af897 P 5bcbfb0f0c194a6caec64d86e9dd07e6: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260808 06:08:36.638450 29537 tablet.cc:2369] T e194f49ccf6647b49cd5b467452af897 P 237be1b81b2f4765bb46cec0037d92c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 06:08:36.682713 29326 tablet.cc:2369] T e194f49ccf6647b49cd5b467452af897 P 5bcbfb0f0c194a6caec64d86e9dd07e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 06:08:36.688400 29575 raft_consensus.cc:482] T e194f49ccf6647b49cd5b467452af897 P 5bcbfb0f0c194a6caec64d86e9dd07e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:08:36.688592 29575 raft_consensus.cc:504] T e194f49ccf6647b49cd5b467452af897 P 5bcbfb0f0c194a6caec64d86e9dd07e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "237be1b81b2f4765bb46cec0037d92c3" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 37135 } } peers { permanent_uuid: "ae9d8d8d731e4f05bbb368fc611af1e6" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 46795 } } peers { permanent_uuid: "5bcbfb0f0c194a6caec64d86e9dd07e6" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 43521 } }
I20260808 06:08:36.689747 29575 leader_election.cc:288] T e194f49ccf6647b49cd5b467452af897 P 5bcbfb0f0c194a6caec64d86e9dd07e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 237be1b81b2f4765bb46cec0037d92c3 (127.1.151.131:37135), ae9d8d8d731e4f05bbb368fc611af1e6 (127.1.151.129:46795)
I20260808 06:08:36.693598 29330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:36.689976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:37135 (local address 127.1.151.130:53851)
0808 06:08:36.690080 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:36.690096 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:36.690214 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:36.690487 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:36.690491 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:36.690511 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:08:36.690620 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:36.690625 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:36.691519 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:36.691522 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:36.692412 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:36.692422 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:36.693158 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:36.693162 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:36.693268 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:36.693327 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 06:08:36.693395 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 06:08:36.693456 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24,"mutex_wait_us":25}
I20260808 06:08:36.693665 29577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:36.689975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:46795 (local address 127.1.151.130:49801)
0808 06:08:36.690572 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:36.690583 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:36.690676 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:36.690887 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:36.690890 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:36.690907 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:08:36.691022 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:36.691028 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:36.691772 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:36.691775 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:36.692635 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:36.692647 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:36.693274 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:36.693277 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:36.693372 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:36.693395 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 06:08:36.693468 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 06:08:36.693530 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"mutex_wait_us":333,"thread_start_us":102,"threads_started":1}
I20260808 06:08:36.693655 29562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:36.690099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:53851 (local address 127.1.151.131:37135)
0808 06:08:36.690263 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:36.690267 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:36.690280 (+ 13us) server_negotiation.cc:407] Connection header received
0808 06:08:36.690332 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:36.690334 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:36.690374 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:36.690451 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:08:36.690765 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:36.691409 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:36.692544 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:36.693023 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:36.693116 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:36.693158 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:36.693431 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 06:08:36.693506 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260808 06:08:36.693675 29564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:36.690093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:49801 (local address 127.1.151.129:46795)
0808 06:08:36.690246 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:36.690251 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:36.690698 (+ 447us) server_negotiation.cc:407] Connection header received
0808 06:08:36.690760 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:36.690762 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:36.690808 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:36.690863 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:08:36.691113 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:36.691663 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:36.692760 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:36.693165 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:36.693277 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:36.693321 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:36.693496 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 06:08:36.693561 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 06:08:36.694269 29466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e194f49ccf6647b49cd5b467452af897" candidate_uuid: "5bcbfb0f0c194a6caec64d86e9dd07e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "237be1b81b2f4765bb46cec0037d92c3" is_pre_election: true
I20260808 06:08:36.694314 29048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e194f49ccf6647b49cd5b467452af897" candidate_uuid: "5bcbfb0f0c194a6caec64d86e9dd07e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae9d8d8d731e4f05bbb368fc611af1e6" is_pre_election: true
I20260808 06:08:36.694541 29466 raft_consensus.cc:2454] T e194f49ccf6647b49cd5b467452af897 P 237be1b81b2f4765bb46cec0037d92c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5bcbfb0f0c194a6caec64d86e9dd07e6 in term 0.
I20260808 06:08:36.694555 29048 raft_consensus.cc:2454] T e194f49ccf6647b49cd5b467452af897 P ae9d8d8d731e4f05bbb368fc611af1e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5bcbfb0f0c194a6caec64d86e9dd07e6 in term 0.
I20260808 06:08:36.695003 29149 leader_election.cc:302] T e194f49ccf6647b49cd5b467452af897 P 5bcbfb0f0c194a6caec64d86e9dd07e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 237be1b81b2f4765bb46cec0037d92c3, 5bcbfb0f0c194a6caec64d86e9dd07e6; no voters:
I20260808 06:08:36.695271 29575 raft_consensus.cc:2781] T e194f49ccf6647b49cd5b467452af897 P 5bcbfb0f0c194a6caec64d86e9dd07e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 06:08:36.695346 29575 raft_consensus.cc:482] T e194f49ccf6647b49cd5b467452af897 P 5bcbfb0f0c194a6caec64d86e9dd07e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 06:08:36.695377 29575 raft_consensus.cc:3037] T e194f49ccf6647b49cd5b467452af897 P 5bcbfb0f0c194a6caec64d86e9dd07e6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:08:36.696422 29575 raft_consensus.cc:504] T e194f49ccf6647b49cd5b467452af897 P 5bcbfb0f0c194a6caec64d86e9dd07e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "237be1b81b2f4765bb46cec0037d92c3" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 37135 } } peers { permanent_uuid: "ae9d8d8d731e4f05bbb368fc611af1e6" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 46795 } } peers { permanent_uuid: "5bcbfb0f0c194a6caec64d86e9dd07e6" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 43521 } }
I20260808 06:08:36.696745 29575 leader_election.cc:288] T e194f49ccf6647b49cd5b467452af897 P 5bcbfb0f0c194a6caec64d86e9dd07e6 [CANDIDATE]: Term 1 election: Requested vote from peers 237be1b81b2f4765bb46cec0037d92c3 (127.1.151.131:37135), ae9d8d8d731e4f05bbb368fc611af1e6 (127.1.151.129:46795)
I20260808 06:08:36.697072 29466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e194f49ccf6647b49cd5b467452af897" candidate_uuid: "5bcbfb0f0c194a6caec64d86e9dd07e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "237be1b81b2f4765bb46cec0037d92c3"
I20260808 06:08:36.697088 29048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e194f49ccf6647b49cd5b467452af897" candidate_uuid: "5bcbfb0f0c194a6caec64d86e9dd07e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae9d8d8d731e4f05bbb368fc611af1e6"
I20260808 06:08:36.697173 29466 raft_consensus.cc:3037] T e194f49ccf6647b49cd5b467452af897 P 237be1b81b2f4765bb46cec0037d92c3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:08:36.697238 29048 raft_consensus.cc:3037] T e194f49ccf6647b49cd5b467452af897 P ae9d8d8d731e4f05bbb368fc611af1e6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:08:36.698309 29466 raft_consensus.cc:2454] T e194f49ccf6647b49cd5b467452af897 P 237be1b81b2f4765bb46cec0037d92c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5bcbfb0f0c194a6caec64d86e9dd07e6 in term 1.
I20260808 06:08:36.698503 29048 raft_consensus.cc:2454] T e194f49ccf6647b49cd5b467452af897 P ae9d8d8d731e4f05bbb368fc611af1e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5bcbfb0f0c194a6caec64d86e9dd07e6 in term 1.
I20260808 06:08:36.698608 29149 leader_election.cc:302] T e194f49ccf6647b49cd5b467452af897 P 5bcbfb0f0c194a6caec64d86e9dd07e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 237be1b81b2f4765bb46cec0037d92c3, 5bcbfb0f0c194a6caec64d86e9dd07e6; no voters:
I20260808 06:08:36.698761 29575 raft_consensus.cc:2781] T e194f49ccf6647b49cd5b467452af897 P 5bcbfb0f0c194a6caec64d86e9dd07e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:08:36.698972 29575 raft_consensus.cc:686] T e194f49ccf6647b49cd5b467452af897 P 5bcbfb0f0c194a6caec64d86e9dd07e6 [term 1 LEADER]: Becoming Leader. State: Replica: 5bcbfb0f0c194a6caec64d86e9dd07e6, State: Running, Role: LEADER
I20260808 06:08:36.699297 29575 consensus_queue.cc:237] T e194f49ccf6647b49cd5b467452af897 P 5bcbfb0f0c194a6caec64d86e9dd07e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "237be1b81b2f4765bb46cec0037d92c3" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 37135 } } peers { permanent_uuid: "ae9d8d8d731e4f05bbb368fc611af1e6" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 46795 } } peers { permanent_uuid: "5bcbfb0f0c194a6caec64d86e9dd07e6" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 43521 } }
I20260808 06:08:36.702366 28816 catalog_manager.cc:5495] T e194f49ccf6647b49cd5b467452af897 P 5bcbfb0f0c194a6caec64d86e9dd07e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5bcbfb0f0c194a6caec64d86e9dd07e6 (127.1.151.130). New cstate: current_term: 1 leader_uuid: "5bcbfb0f0c194a6caec64d86e9dd07e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "237be1b81b2f4765bb46cec0037d92c3" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 37135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae9d8d8d731e4f05bbb368fc611af1e6" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 46795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5bcbfb0f0c194a6caec64d86e9dd07e6" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 43521 } health_report { overall_health: HEALTHY } } }
I20260808 06:08:36.732362 29566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:36.728971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38814 (local address 127.1.151.130:43521)
0808 06:08:36.729091 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:36.729096 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:36.729117 (+ 21us) server_negotiation.cc:407] Connection header received
0808 06:08:36.729231 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:36.729234 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:36.729285 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:36.729369 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:08:36.729616 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:36.730184 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:36.731026 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:36.731249 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:36.731333 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:36.731614 (+ 281us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:08:36.731993 (+ 379us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:08:36.732106 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:36.732176 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 06:08:36.732235 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 06:08:36.738274 29466 raft_consensus.cc:1261] T e194f49ccf6647b49cd5b467452af897 P 237be1b81b2f4765bb46cec0037d92c3 [term 1 FOLLOWER]: Refusing update from remote peer 5bcbfb0f0c194a6caec64d86e9dd07e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:08:36.738273 29048 raft_consensus.cc:1261] T e194f49ccf6647b49cd5b467452af897 P ae9d8d8d731e4f05bbb368fc611af1e6 [term 1 FOLLOWER]: Refusing update from remote peer 5bcbfb0f0c194a6caec64d86e9dd07e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:08:36.739014 29575 consensus_queue.cc:1035] T e194f49ccf6647b49cd5b467452af897 P 5bcbfb0f0c194a6caec64d86e9dd07e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "237be1b81b2f4765bb46cec0037d92c3" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:08:36.739213 29578 consensus_queue.cc:1035] T e194f49ccf6647b49cd5b467452af897 P 5bcbfb0f0c194a6caec64d86e9dd07e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae9d8d8d731e4f05bbb368fc611af1e6" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 46795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:08:36.747412 29586 mvcc.cc:205] Tried to move back new op lower bound from 7316149521352364032 to 7316149521202802688. Current Snapshot: MvccSnapshot[applied={T|T < 7316149521352364032}]
I20260808 06:08:36.748644 29585 mvcc.cc:205] Tried to move back new op lower bound from 7316149521352364032 to 7316149521202802688. Current Snapshot: MvccSnapshot[applied={T|T < 7316149521352364032}]
I20260808 06:08:36.754132 28814 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:08:36.754314 28814 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:08:36.756608 28814 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 67115259c1224700be14e9ca16775b5d: Sending DeleteTablet for 3 replicas of tablet e194f49ccf6647b49cd5b467452af897
I20260808 06:08:36.757309 29434 tablet_service.cc:1514] Processing DeleteTablet for tablet e194f49ccf6647b49cd5b467452af897 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:08:36 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:38158
I20260808 06:08:36.757351 29015 tablet_service.cc:1514] Processing DeleteTablet for tablet e194f49ccf6647b49cd5b467452af897 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:08:36 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:36204
I20260808 06:08:36.757947 29599 tablet_replica.cc:330] T e194f49ccf6647b49cd5b467452af897 P ae9d8d8d731e4f05bbb368fc611af1e6: stopping tablet replica
I20260808 06:08:36.757926 29220 tablet_service.cc:1514] Processing DeleteTablet for tablet e194f49ccf6647b49cd5b467452af897 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:08:36 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:38812
I20260808 06:08:36.758283 29599 raft_consensus.cc:2229] T e194f49ccf6647b49cd5b467452af897 P ae9d8d8d731e4f05bbb368fc611af1e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:08:36.758452 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 28915
I20260808 06:08:36.758574 29599 raft_consensus.cc:2258] T e194f49ccf6647b49cd5b467452af897 P ae9d8d8d731e4f05bbb368fc611af1e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:08:36.758968 29598 tablet_replica.cc:330] T e194f49ccf6647b49cd5b467452af897 P 237be1b81b2f4765bb46cec0037d92c3: stopping tablet replica
I20260808 06:08:36.759238 29598 raft_consensus.cc:2229] T e194f49ccf6647b49cd5b467452af897 P 237be1b81b2f4765bb46cec0037d92c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:08:36.759485 29598 raft_consensus.cc:2258] T e194f49ccf6647b49cd5b467452af897 P 237be1b81b2f4765bb46cec0037d92c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:08:36.762624 29598 ts_tablet_manager.cc:1905] T e194f49ccf6647b49cd5b467452af897 P 237be1b81b2f4765bb46cec0037d92c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:08:36.765380 29598 ts_tablet_manager.cc:1918] T e194f49ccf6647b49cd5b467452af897 P 237be1b81b2f4765bb46cec0037d92c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:08:36.765479 29598 log.cc:1196] T e194f49ccf6647b49cd5b467452af897 P 237be1b81b2f4765bb46cec0037d92c3: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786169070948234-1630-0/minicluster-data/ts-2/wal/wals/e194f49ccf6647b49cd5b467452af897
I20260808 06:08:36.765987 29598 ts_tablet_manager.cc:1939] T e194f49ccf6647b49cd5b467452af897 P 237be1b81b2f4765bb46cec0037d92c3: Deleting consensus metadata
I20260808 06:08:36.767210 28792 catalog_manager.cc:4841] TS 237be1b81b2f4765bb46cec0037d92c3 (127.1.151.131:37135): tablet e194f49ccf6647b49cd5b467452af897 (table test-table [id=94fc8bc9d28448dfa5134e3dffbb0023]) successfully deleted
I20260808 06:08:36.767329 29600 tablet_replica.cc:330] T e194f49ccf6647b49cd5b467452af897 P 5bcbfb0f0c194a6caec64d86e9dd07e6: stopping tablet replica
I20260808 06:08:36.767587 29600 raft_consensus.cc:2229] T e194f49ccf6647b49cd5b467452af897 P 5bcbfb0f0c194a6caec64d86e9dd07e6 [term 1 LEADER]: Raft consensus shutting down.
I20260808 06:08:36.767947 29600 raft_consensus.cc:2258] T e194f49ccf6647b49cd5b467452af897 P 5bcbfb0f0c194a6caec64d86e9dd07e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:08:36.769382 29600 ts_tablet_manager.cc:1905] T e194f49ccf6647b49cd5b467452af897 P 5bcbfb0f0c194a6caec64d86e9dd07e6: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 06:08:36.769596 28795 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.151.129:46795 (error 108)
I20260808 06:08:36.770593 29563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:36.770321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:46795 (local address 127.0.0.1:36220)
0808 06:08:36.770443 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:36.770512 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.129:46795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 06:08:36.770692 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 29121
W20260808 06:08:36.776300 28795 catalog_manager.cc:4584] TS ae9d8d8d731e4f05bbb368fc611af1e6 (127.1.151.129:46795): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e194f49ccf6647b49cd5b467452af897: Network error: Client connection negotiation failed: client connection to 127.1.151.129:46795: connect: Connection refused (error 111)
I20260808 06:08:36.782588 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 29333
I20260808 06:08:36.784300 29563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:36.782698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:43521 (local address 127.0.0.1:38824)
0808 06:08:36.784173 (+ 1475us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:36.784228 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.130:43521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1384}
I20260808 06:08:36.792477 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 28763
2026-08-08T06:08:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2576 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T06:08:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:08:36Z Disabled control of system clock
I20260808 06:08:36.828938 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:33715
--webserver_interface=127.1.151.190
--webserver_port=0
--builtin_ntp_servers=127.1.151.148:37037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:33715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 06:08:36.937691 29609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:36.937937 29609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:36.937970 29609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:36.941911 29609 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:08:36.941969 29609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:36.941983 29609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:36.941998 29609 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:08:36.942013 29609 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:08:36.946750 29609 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:37037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:33715
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:33715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:36.947834 29609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:36.949198 29609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:08:36.957367 29629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:36.957360 29625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:36.957360 29624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:36.957518 29609 server_base.cc:1034] running on GCE node
I20260808 06:08:36.958307 29609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:36.959323 29609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:36.960471 29609 hybrid_clock.cc:648] HybridClock initialized: now 1786169316960448 us; error 34 us; skew 500 ppm
I20260808 06:08:36.962215 29609 webserver.cc:460] Webserver started at http://127.1.151.190:44747/ using document root <none> and password file <none>
I20260808 06:08:36.962766 29609 fs_manager.cc:359] Metadata directory not provided
I20260808 06:08:36.962828 29609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:08:36.963017 29609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:08:36.964944 29609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "5586251030f940e6b4b4243d1c811c03"
format_stamp: "Formatted at 2026-08-08 06:08:36 on dist-test-slave-zj8q"
server_key: "6064c93e62671e6ee655718442cd5512"
server_key_iv: "2523c9b5ef366e8d04ecff71ce0a568f"
server_key_version: "encryptionkey@0"
I20260808 06:08:36.965500 29609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "5586251030f940e6b4b4243d1c811c03"
format_stamp: "Formatted at 2026-08-08 06:08:36 on dist-test-slave-zj8q"
server_key: "6064c93e62671e6ee655718442cd5512"
server_key_iv: "2523c9b5ef366e8d04ecff71ce0a568f"
server_key_version: "encryptionkey@0"
I20260808 06:08:36.968964 29609 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 06:08:36.971235 29644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:36.972359 29609 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 06:08:36.972470 29609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "5586251030f940e6b4b4243d1c811c03"
format_stamp: "Formatted at 2026-08-08 06:08:36 on dist-test-slave-zj8q"
server_key: "6064c93e62671e6ee655718442cd5512"
server_key_iv: "2523c9b5ef366e8d04ecff71ce0a568f"
server_key_version: "encryptionkey@0"
I20260808 06:08:36.972555 29609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:08:36.986091 29609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:08:36.986697 29609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:08:36.986852 29609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:08:36.993250 29609 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:33715
I20260808 06:08:36.993255 29719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:33715 every 8 connection(s)
I20260808 06:08:36.994092 29609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:08:36.994280 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 29609
I20260808 06:08:36.994395 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:08:36.994679 1630 external_mini_cluster.cc:1428] Setting key 4a4ee314484d3444cc7f5bae68e77f38
I20260808 06:08:36.997298 29720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:08:37.003485 29720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5586251030f940e6b4b4243d1c811c03: Bootstrap starting.
I20260808 06:08:37.004002 29725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:36.996019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40974 (local address 127.1.151.190:33715)
0808 06:08:36.996663 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:36.996678 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:36.996711 (+ 33us) server_negotiation.cc:407] Connection header received
0808 06:08:36.997324 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:36.997342 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:36.997741 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:36.997998 (+ 257us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:36.998395 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:36.999481 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.000724 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:37.001182 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.001298 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:37.002185 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:37.002193 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:37.002210 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:08:37.002633 (+ 423us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:37.002693 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:08:37.002718 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:37.002801 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:37.002953 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 06:08:37.003211 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":140,"threads_started":1}
I20260808 06:08:37.005666 29720 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5586251030f940e6b4b4243d1c811c03: Neither blocks nor log segments found. Creating new log.
I20260808 06:08:37.006460 29720 log.cc:824] T 00000000000000000000000000000000 P 5586251030f940e6b4b4243d1c811c03: Log is configured to *not* fsync() on all Append() calls
I20260808 06:08:37.008200 29720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5586251030f940e6b4b4243d1c811c03: No bootstrap required, opened a new log
I20260808 06:08:37.011564 29720 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5586251030f940e6b4b4243d1c811c03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5586251030f940e6b4b4243d1c811c03" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 33715 } }
I20260808 06:08:37.011775 29720 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5586251030f940e6b4b4243d1c811c03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:08:37.011808 29720 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5586251030f940e6b4b4243d1c811c03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5586251030f940e6b4b4243d1c811c03, State: Initialized, Role: FOLLOWER
I20260808 06:08:37.012144 29720 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5586251030f940e6b4b4243d1c811c03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5586251030f940e6b4b4243d1c811c03" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 33715 } }
I20260808 06:08:37.012251 29720 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5586251030f940e6b4b4243d1c811c03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:08:37.012292 29720 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5586251030f940e6b4b4243d1c811c03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:08:37.012358 29720 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5586251030f940e6b4b4243d1c811c03 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:08:37.013187 29720 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5586251030f940e6b4b4243d1c811c03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5586251030f940e6b4b4243d1c811c03" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 33715 } }
I20260808 06:08:37.013469 29720 leader_election.cc:302] T 00000000000000000000000000000000 P 5586251030f940e6b4b4243d1c811c03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5586251030f940e6b4b4243d1c811c03; no voters:
I20260808 06:08:37.013717 29720 leader_election.cc:288] T 00000000000000000000000000000000 P 5586251030f940e6b4b4243d1c811c03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:08:37.013824 29728 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5586251030f940e6b4b4243d1c811c03 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:08:37.014030 29728 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5586251030f940e6b4b4243d1c811c03 [term 1 LEADER]: Becoming Leader. State: Replica: 5586251030f940e6b4b4243d1c811c03, State: Running, Role: LEADER
I20260808 06:08:37.014384 29728 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5586251030f940e6b4b4243d1c811c03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5586251030f940e6b4b4243d1c811c03" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 33715 } }
I20260808 06:08:37.014802 29720 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5586251030f940e6b4b4243d1c811c03 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:08:37.015918 29729 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5586251030f940e6b4b4243d1c811c03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5586251030f940e6b4b4243d1c811c03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5586251030f940e6b4b4243d1c811c03" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 33715 } } }
I20260808 06:08:37.016027 29729 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5586251030f940e6b4b4243d1c811c03 [sys.catalog]: This master's current role is: LEADER
I20260808 06:08:37.016323 29731 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5586251030f940e6b4b4243d1c811c03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5586251030f940e6b4b4243d1c811c03. Latest consensus state: current_term: 1 leader_uuid: "5586251030f940e6b4b4243d1c811c03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5586251030f940e6b4b4243d1c811c03" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 33715 } } }
I20260808 06:08:37.016479 29731 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5586251030f940e6b4b4243d1c811c03 [sys.catalog]: This master's current role is: LEADER
I20260808 06:08:37.016839 29744 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:08:37.020054 29744 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:08:37.025882 29744 catalog_manager.cc:1347] Generated new cluster ID: 596f59336bbb4d3e8930cf7fbcfc3fdc
I20260808 06:08:37.025950 29744 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:08:37.039490 29744 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:08:37.040560 29744 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:08:37.047698 29744 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5586251030f940e6b4b4243d1c811c03: Generated new TSK 0
I20260808 06:08:37.048269 29744 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 06:08:37.053963 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.129:0
--local_ip_for_outbound_sockets=127.1.151.129
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:33715
--builtin_ntp_servers=127.1.151.148:37037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 06:08:37.186053 29759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:37.186355 29759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:37.186414 29759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:37.190445 29759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:37.190493 29759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:37.190564 29759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.129
I20260808 06:08:37.195238 29759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:37037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:33715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.129
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:37.196244 29759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:37.197611 29759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:08:37.206995 29774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:37.206988 29769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:37.206988 29767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:37.207402 29759 server_base.cc:1034] running on GCE node
I20260808 06:08:37.207921 29759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:37.208648 29759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:37.209826 29759 hybrid_clock.cc:648] HybridClock initialized: now 1786169317209812 us; error 29 us; skew 500 ppm
I20260808 06:08:37.211889 29759 webserver.cc:460] Webserver started at http://127.1.151.129:41313/ using document root <none> and password file <none>
I20260808 06:08:37.212477 29759 fs_manager.cc:359] Metadata directory not provided
I20260808 06:08:37.212548 29759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:08:37.212750 29759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:08:37.214862 29759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-0/data/instance:
uuid: "0b1b8ac5edc54a20a4ea3751e8fa26ca"
format_stamp: "Formatted at 2026-08-08 06:08:37 on dist-test-slave-zj8q"
server_key: "2d4fc9a011624ca03fda49ea05ed6464"
server_key_iv: "1d290f954f1e5bfef16248fdb24f665d"
server_key_version: "encryptionkey@0"
I20260808 06:08:37.215528 29759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance:
uuid: "0b1b8ac5edc54a20a4ea3751e8fa26ca"
format_stamp: "Formatted at 2026-08-08 06:08:37 on dist-test-slave-zj8q"
server_key: "2d4fc9a011624ca03fda49ea05ed6464"
server_key_iv: "1d290f954f1e5bfef16248fdb24f665d"
server_key_version: "encryptionkey@0"
I20260808 06:08:37.219187 29759 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 06:08:37.221480 29782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:37.222671 29759 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:08:37.222779 29759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-0/wal
uuid: "0b1b8ac5edc54a20a4ea3751e8fa26ca"
format_stamp: "Formatted at 2026-08-08 06:08:37 on dist-test-slave-zj8q"
server_key: "2d4fc9a011624ca03fda49ea05ed6464"
server_key_iv: "1d290f954f1e5bfef16248fdb24f665d"
server_key_version: "encryptionkey@0"
I20260808 06:08:37.222870 29759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:08:37.238116 29759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:08:37.238799 29759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:08:37.238965 29759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:08:37.239537 29759 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:08:37.240466 29759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:08:37.240513 29759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:37.240564 29759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:08:37.240594 29759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:37.249343 29759 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.129:35615
I20260808 06:08:37.249370 29971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.129:35615 every 8 connection(s)
I20260808 06:08:37.250242 29759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
I20260808 06:08:37.252844 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 29759
I20260808 06:08:37.252960 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance
I20260808 06:08:37.253293 1630 external_mini_cluster.cc:1428] Setting key 0765e38a3b48668a15f063c02fc74e4e
I20260808 06:08:37.255307 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.130:0
--local_ip_for_outbound_sockets=127.1.151.130
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:33715
--builtin_ntp_servers=127.1.151.148:37037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 06:08:37.259137 29725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:37.251971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:56717 (local address 127.1.151.190:33715)
0808 06:08:37.252093 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:37.252097 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:37.253100 (+ 1003us) server_negotiation.cc:407] Connection header received
0808 06:08:37.254653 (+ 1553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:37.254656 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:37.254708 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:37.254779 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:37.255678 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:37.256209 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.257333 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:37.257559 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.257653 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:37.258654 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:37.258658 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:37.258661 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:08:37.258674 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:37.258694 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:08:37.258702 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:37.258761 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:37.258905 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 06:08:37.258991 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 06:08:37.259940 29976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:37.252317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:33715 (local address 127.1.151.129:56717)
0808 06:08:37.252934 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:37.252987 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:37.254438 (+ 1451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:37.254940 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:37.254946 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:37.255026 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:08:37.255542 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:37.255559 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:37.256332 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.256335 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:37.257212 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:37.257222 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:37.257699 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.257704 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:37.257859 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:37.258408 (+ 549us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 06:08:37.258410 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:08:37.258447 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 06:08:37.258507 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:08:37.258773 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:08:37.258783 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:08:37.258789 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 06:08:37.258892 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 06:08:37.259171 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":113,"threads_started":1}
I20260808 06:08:37.261227 29974 heartbeater.cc:347] Connected to a master server at 127.1.151.190:33715
I20260808 06:08:37.261428 29974 heartbeater.cc:464] Registering TS with master...
I20260808 06:08:37.261883 29974 heartbeater.cc:511] Master 127.1.151.190:33715 requested a full tablet report, sending...
I20260808 06:08:37.263617 29669 ts_manager.cc:194] Registered new tserver with Master: 0b1b8ac5edc54a20a4ea3751e8fa26ca (127.1.151.129:35615)
I20260808 06:08:37.265280 29669 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.151.129:56717
W20260808 06:08:37.367353 29977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:37.367578 29977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:37.367611 29977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:37.371431 29977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:37.371474 29977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:37.371522 29977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.130
I20260808 06:08:37.376071 29977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:37037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:33715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.130
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:37.377044 29977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:37.378324 29977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:08:37.387511 29985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:37.387645 29977 server_base.cc:1034] running on GCE node
W20260808 06:08:37.387545 29986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:37.387545 29988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:37.388140 29977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:37.388746 29977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:37.389907 29977 hybrid_clock.cc:648] HybridClock initialized: now 1786169317389892 us; error 27 us; skew 500 ppm
I20260808 06:08:37.391702 29977 webserver.cc:460] Webserver started at http://127.1.151.130:34123/ using document root <none> and password file <none>
I20260808 06:08:37.392251 29977 fs_manager.cc:359] Metadata directory not provided
I20260808 06:08:37.392316 29977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:08:37.392499 29977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:08:37.394539 29977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-1/data/instance:
uuid: "2cc901283a4e4ae593572fc97ab181d2"
format_stamp: "Formatted at 2026-08-08 06:08:37 on dist-test-slave-zj8q"
server_key: "0146ca57244bf4302f6674bdcf8dd71b"
server_key_iv: "89fb31201f655a30946c68682057a6f1"
server_key_version: "encryptionkey@0"
I20260808 06:08:37.395079 29977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance:
uuid: "2cc901283a4e4ae593572fc97ab181d2"
format_stamp: "Formatted at 2026-08-08 06:08:37 on dist-test-slave-zj8q"
server_key: "0146ca57244bf4302f6674bdcf8dd71b"
server_key_iv: "89fb31201f655a30946c68682057a6f1"
server_key_version: "encryptionkey@0"
I20260808 06:08:37.398571 29977 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 06:08:37.400903 29999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:37.402102 29977 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:08:37.402251 29977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-1/wal
uuid: "2cc901283a4e4ae593572fc97ab181d2"
format_stamp: "Formatted at 2026-08-08 06:08:37 on dist-test-slave-zj8q"
server_key: "0146ca57244bf4302f6674bdcf8dd71b"
server_key_iv: "89fb31201f655a30946c68682057a6f1"
server_key_version: "encryptionkey@0"
I20260808 06:08:37.402350 29977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:08:37.415473 29977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:08:37.416085 29977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:08:37.416244 29977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:08:37.417644 29977 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:08:37.418567 29977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:08:37.418615 29977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:37.418676 29977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:08:37.418700 29977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:37.427054 29977 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.130:44555
I20260808 06:08:37.427081 30173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.130:44555 every 8 connection(s)
I20260808 06:08:37.427891 29977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
I20260808 06:08:37.432120 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 29977
I20260808 06:08:37.432219 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance
I20260808 06:08:37.432533 1630 external_mini_cluster.cc:1428] Setting key 2b6ce07d0e61de1a054c5e97e5a7fd31
I20260808 06:08:37.434635 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.131:0
--local_ip_for_outbound_sockets=127.1.151.131
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:33715
--builtin_ntp_servers=127.1.151.148:37037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 06:08:37.437376 29725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:37.429778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:40055 (local address 127.1.151.190:33715)
0808 06:08:37.429941 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:37.429946 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:37.431004 (+ 1058us) server_negotiation.cc:407] Connection header received
0808 06:08:37.432392 (+ 1388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:37.432396 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:37.432456 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:37.432559 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:37.433573 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:37.434231 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.435484 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:37.435773 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.435876 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:37.436956 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:37.436959 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:37.436961 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:08:37.436974 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:37.436994 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:08:37.437002 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:37.437069 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:37.437213 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 06:08:37.437266 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 06:08:37.438346 30177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:37.430176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:33715 (local address 127.1.151.130:40055)
0808 06:08:37.430859 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:37.430899 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:37.432157 (+ 1258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:37.432713 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:37.432720 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:37.432801 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:08:37.433397 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:37.433418 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:37.434355 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.434358 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:37.435362 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:37.435373 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:37.435921 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.435924 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:37.436070 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:37.436671 (+ 601us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 06:08:37.436673 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:08:37.436725 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 06:08:37.436802 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:08:37.437076 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:08:37.437085 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:08:37.437090 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 06:08:37.437215 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 06:08:37.437488 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":113,"threads_started":1}
I20260808 06:08:37.439394 30174 heartbeater.cc:347] Connected to a master server at 127.1.151.190:33715
I20260808 06:08:37.439642 30174 heartbeater.cc:464] Registering TS with master...
I20260808 06:08:37.440245 30174 heartbeater.cc:511] Master 127.1.151.190:33715 requested a full tablet report, sending...
I20260808 06:08:37.441345 29669 ts_manager.cc:194] Registered new tserver with Master: 2cc901283a4e4ae593572fc97ab181d2 (127.1.151.130:44555)
I20260808 06:08:37.441888 29669 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.151.130:40055
W20260808 06:08:37.548048 30179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:37.548300 30179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:37.548341 30179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:37.552386 30179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:37.552445 30179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:37.552505 30179 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.131
I20260808 06:08:37.557343 30179 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:37037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:33715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.131
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:37.558498 30179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:37.559887 30179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:08:37.569665 30189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:37.569691 30185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:37.569713 30179 server_base.cc:1034] running on GCE node
W20260808 06:08:37.569762 30186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:37.570325 30179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:37.570928 30179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:37.572090 30179 hybrid_clock.cc:648] HybridClock initialized: now 1786169317572070 us; error 31 us; skew 500 ppm
I20260808 06:08:37.573882 30179 webserver.cc:460] Webserver started at http://127.1.151.131:46769/ using document root <none> and password file <none>
I20260808 06:08:37.574484 30179 fs_manager.cc:359] Metadata directory not provided
I20260808 06:08:37.574550 30179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:08:37.574806 30179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:08:37.576984 30179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-2/data/instance:
uuid: "6192b6938bfd4cb49a7f1f5b0e13106c"
format_stamp: "Formatted at 2026-08-08 06:08:37 on dist-test-slave-zj8q"
server_key: "68374a4b12ebbff9e1be037d6120dac0"
server_key_iv: "8404fea00275a37ed4620635557b0033"
server_key_version: "encryptionkey@0"
I20260808 06:08:37.577528 30179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance:
uuid: "6192b6938bfd4cb49a7f1f5b0e13106c"
format_stamp: "Formatted at 2026-08-08 06:08:37 on dist-test-slave-zj8q"
server_key: "68374a4b12ebbff9e1be037d6120dac0"
server_key_iv: "8404fea00275a37ed4620635557b0033"
server_key_version: "encryptionkey@0"
I20260808 06:08:37.581166 30179 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.002s
I20260808 06:08:37.583532 30196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:37.584774 30179 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 06:08:37.584884 30179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-2/wal
uuid: "6192b6938bfd4cb49a7f1f5b0e13106c"
format_stamp: "Formatted at 2026-08-08 06:08:37 on dist-test-slave-zj8q"
server_key: "68374a4b12ebbff9e1be037d6120dac0"
server_key_iv: "8404fea00275a37ed4620635557b0033"
server_key_version: "encryptionkey@0"
I20260808 06:08:37.584971 30179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:08:37.619333 30179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:08:37.620069 30179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:08:37.620244 30179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:08:37.620827 30179 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:08:37.621796 30179 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:08:37.621845 30179 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:37.621886 30179 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:08:37.621906 30179 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:37.630239 30179 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.131:33541
I20260808 06:08:37.630287 30378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.131:33541 every 8 connection(s)
I20260808 06:08:37.631134 30179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
I20260808 06:08:37.632313 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 30179
I20260808 06:08:37.632426 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance
I20260808 06:08:37.632735 1630 external_mini_cluster.cc:1428] Setting key 421d606138c195d3cb9429574b0af0ea
I20260808 06:08:37.640435 29725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:37.633273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:57513 (local address 127.1.151.190:33715)
0808 06:08:37.633418 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:37.633422 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:37.634437 (+ 1015us) server_negotiation.cc:407] Connection header received
0808 06:08:37.635978 (+ 1541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:37.635981 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:37.636027 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:37.636090 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:37.636964 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:37.637496 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.638777 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:37.638999 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.639083 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:37.639996 (+ 913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:37.639999 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:37.640001 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:08:37.640013 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:37.640033 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:08:37.640041 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:37.640095 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:37.640274 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 06:08:37.640325 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 06:08:37.641335 30383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:37.633516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:33715 (local address 127.1.151.131:57513)
0808 06:08:37.634295 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:37.634338 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:37.635796 (+ 1458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:37.636227 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:37.636233 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:37.636313 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:08:37.636808 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:37.636821 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:37.637673 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.637676 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:37.638648 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:37.638659 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:37.639119 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.639121 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:37.639260 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:37.639767 (+ 507us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 06:08:37.639768 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:08:37.639806 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 06:08:37.639868 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:08:37.640137 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:08:37.640145 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:08:37.640150 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 06:08:37.640272 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 06:08:37.640489 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":125,"threads_started":1}
I20260808 06:08:37.642351 30379 heartbeater.cc:347] Connected to a master server at 127.1.151.190:33715
I20260808 06:08:37.642572 30379 heartbeater.cc:464] Registering TS with master...
I20260808 06:08:37.643085 30379 heartbeater.cc:511] Master 127.1.151.190:33715 requested a full tablet report, sending...
I20260808 06:08:37.644037 29669 ts_manager.cc:194] Registered new tserver with Master: 6192b6938bfd4cb49a7f1f5b0e13106c (127.1.151.131:33541)
I20260808 06:08:37.644577 29669 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.151.131:57513
I20260808 06:08:37.646925 1630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 06:08:37.652151 29725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:37.648793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40978 (local address 127.1.151.190:33715)
0808 06:08:37.648886 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:37.648889 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:37.649097 (+ 208us) server_negotiation.cc:407] Connection header received
0808 06:08:37.649195 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:37.649197 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:37.649231 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:37.649292 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:37.649489 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:37.649963 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.650864 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:37.651096 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.651181 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:37.651839 (+ 658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:37.651843 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:37.651845 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:08:37.651855 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:37.651872 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:08:37.651880 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:37.651936 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:37.651995 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 06:08:37.652051 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 06:08:37.654832 29669 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40978:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 06:08:37.657779 29669 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 06:08:37.672101 30401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:37.666916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:35615 (local address 127.0.0.1:46556)
0808 06:08:37.667466 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:37.667478 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:37.667573 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:37.668219 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:37.668222 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:37.668239 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:08:37.668327 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:37.668333 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:37.669675 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.669679 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:37.670803 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:37.670813 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:37.671576 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.671579 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:37.671693 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:37.671731 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 06:08:37.671834 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 06:08:37.671950 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":93,"threads_started":1}
I20260808 06:08:37.672304 30406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:37.667337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46556 (local address 127.1.151.129:35615)
0808 06:08:37.667825 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:37.667836 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:37.667859 (+ 23us) server_negotiation.cc:407] Connection header received
0808 06:08:37.667928 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:37.667933 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:37.668105 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:37.668240 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:08:37.668435 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:37.669551 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.670921 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:37.671458 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.671559 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:37.671758 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:37.672042 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 06:08:37.672155 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":126,"threads_started":1}
I20260808 06:08:37.672470 30400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:37.666353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47528 (local address 127.1.151.130:44555)
0808 06:08:37.666743 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:37.666754 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:37.667014 (+ 260us) server_negotiation.cc:407] Connection header received
0808 06:08:37.667340 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:37.667346 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:37.667565 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:37.667699 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:08:37.668210 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:37.669119 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.670675 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:37.671469 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.671567 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:37.671763 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:37.672218 (+ 455us) server_negotiation.cc:299] Negotiation successful
0808 06:08:37.672320 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":140,"threads_started":1}
I20260808 06:08:37.672676 30402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:37.666994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43056 (local address 127.1.151.131:33541)
0808 06:08:37.667818 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:37.667830 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:37.667854 (+ 24us) server_negotiation.cc:407] Connection header received
0808 06:08:37.667930 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:37.667936 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:37.668082 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:37.668203 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:08:37.668621 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:37.669703 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.670963 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:37.671704 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.671807 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:37.671969 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:37.672432 (+ 463us) server_negotiation.cc:299] Negotiation successful
0808 06:08:37.672526 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":716,"thread_start_us":164,"threads_started":1}
I20260808 06:08:37.672883 30396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:37.666185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:44555 (local address 127.0.0.1:47528)
0808 06:08:37.666872 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:37.666922 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:37.667145 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:37.667915 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:37.667922 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:37.667945 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:08:37.668070 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:37.668077 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:37.669225 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.669228 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:37.670530 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:37.670542 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:37.671860 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.671864 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:37.672045 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:37.672066 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 06:08:37.672686 (+ 620us) client_negotiation.cc:240] Negotiation successful
0808 06:08:37.672766 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"mutex_wait_us":101,"thread_start_us":88,"threads_started":1}
I20260808 06:08:37.672333 30397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:37.666316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:33541 (local address 127.0.0.1:43056)
0808 06:08:37.667069 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:37.667083 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:37.667181 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:37.668356 (+ 1175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:37.668359 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:37.668377 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:08:37.668479 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:37.668485 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:37.669805 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.669808 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:37.670837 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:37.670847 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:37.671836 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.671840 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:37.671933 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:37.672047 (+ 114us) client_negotiation.cc:769] Sending connection context
0808 06:08:37.672118 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 06:08:37.672193 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"mutex_wait_us":41,"thread_start_us":125,"threads_started":1}
I20260808 06:08:37.675715 30066 tablet_service.cc:1467] Processing CreateTablet for tablet e068ff017692408f8a3d32df2d85bb01 (DEFAULT_TABLE table=test-table [id=80ac5ca83d14452fb4c65569fb9353d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:08:37.675715 29858 tablet_service.cc:1467] Processing CreateTablet for tablet e068ff017692408f8a3d32df2d85bb01 (DEFAULT_TABLE table=test-table [id=80ac5ca83d14452fb4c65569fb9353d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:08:37.676913 30066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e068ff017692408f8a3d32df2d85bb01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:08:37.676911 29858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e068ff017692408f8a3d32df2d85bb01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:08:37.678406 30281 tablet_service.cc:1467] Processing CreateTablet for tablet e068ff017692408f8a3d32df2d85bb01 (DEFAULT_TABLE table=test-table [id=80ac5ca83d14452fb4c65569fb9353d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:08:37.679502 30281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e068ff017692408f8a3d32df2d85bb01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:08:37.684595 30408 tablet_bootstrap.cc:492] T e068ff017692408f8a3d32df2d85bb01 P 0b1b8ac5edc54a20a4ea3751e8fa26ca: Bootstrap starting.
I20260808 06:08:37.685040 30407 tablet_bootstrap.cc:492] T e068ff017692408f8a3d32df2d85bb01 P 2cc901283a4e4ae593572fc97ab181d2: Bootstrap starting.
I20260808 06:08:37.686045 30409 tablet_bootstrap.cc:492] T e068ff017692408f8a3d32df2d85bb01 P 6192b6938bfd4cb49a7f1f5b0e13106c: Bootstrap starting.
I20260808 06:08:37.687414 30407 tablet_bootstrap.cc:654] T e068ff017692408f8a3d32df2d85bb01 P 2cc901283a4e4ae593572fc97ab181d2: Neither blocks nor log segments found. Creating new log.
I20260808 06:08:37.688215 30407 log.cc:824] T e068ff017692408f8a3d32df2d85bb01 P 2cc901283a4e4ae593572fc97ab181d2: Log is configured to *not* fsync() on all Append() calls
I20260808 06:08:37.688297 30408 tablet_bootstrap.cc:654] T e068ff017692408f8a3d32df2d85bb01 P 0b1b8ac5edc54a20a4ea3751e8fa26ca: Neither blocks nor log segments found. Creating new log.
I20260808 06:08:37.688297 30409 tablet_bootstrap.cc:654] T e068ff017692408f8a3d32df2d85bb01 P 6192b6938bfd4cb49a7f1f5b0e13106c: Neither blocks nor log segments found. Creating new log.
I20260808 06:08:37.689146 30409 log.cc:824] T e068ff017692408f8a3d32df2d85bb01 P 6192b6938bfd4cb49a7f1f5b0e13106c: Log is configured to *not* fsync() on all Append() calls
I20260808 06:08:37.689157 30408 log.cc:824] T e068ff017692408f8a3d32df2d85bb01 P 0b1b8ac5edc54a20a4ea3751e8fa26ca: Log is configured to *not* fsync() on all Append() calls
I20260808 06:08:37.690701 30407 tablet_bootstrap.cc:492] T e068ff017692408f8a3d32df2d85bb01 P 2cc901283a4e4ae593572fc97ab181d2: No bootstrap required, opened a new log
I20260808 06:08:37.690898 30407 ts_tablet_manager.cc:1397] T e068ff017692408f8a3d32df2d85bb01 P 2cc901283a4e4ae593572fc97ab181d2: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 06:08:37.691180 30409 tablet_bootstrap.cc:492] T e068ff017692408f8a3d32df2d85bb01 P 6192b6938bfd4cb49a7f1f5b0e13106c: No bootstrap required, opened a new log
I20260808 06:08:37.691195 30408 tablet_bootstrap.cc:492] T e068ff017692408f8a3d32df2d85bb01 P 0b1b8ac5edc54a20a4ea3751e8fa26ca: No bootstrap required, opened a new log
I20260808 06:08:37.691375 30409 ts_tablet_manager.cc:1397] T e068ff017692408f8a3d32df2d85bb01 P 6192b6938bfd4cb49a7f1f5b0e13106c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 06:08:37.691375 30408 ts_tablet_manager.cc:1397] T e068ff017692408f8a3d32df2d85bb01 P 0b1b8ac5edc54a20a4ea3751e8fa26ca: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260808 06:08:37.694253 30407 raft_consensus.cc:348] T e068ff017692408f8a3d32df2d85bb01 P 2cc901283a4e4ae593572fc97ab181d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc901283a4e4ae593572fc97ab181d2" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44555 } } peers { permanent_uuid: "6192b6938bfd4cb49a7f1f5b0e13106c" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33541 } } peers { permanent_uuid: "0b1b8ac5edc54a20a4ea3751e8fa26ca" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35615 } }
I20260808 06:08:37.694491 30407 raft_consensus.cc:374] T e068ff017692408f8a3d32df2d85bb01 P 2cc901283a4e4ae593572fc97ab181d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:08:37.694562 30407 raft_consensus.cc:729] T e068ff017692408f8a3d32df2d85bb01 P 2cc901283a4e4ae593572fc97ab181d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cc901283a4e4ae593572fc97ab181d2, State: Initialized, Role: FOLLOWER
I20260808 06:08:37.694989 30407 consensus_queue.cc:260] T e068ff017692408f8a3d32df2d85bb01 P 2cc901283a4e4ae593572fc97ab181d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc901283a4e4ae593572fc97ab181d2" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44555 } } peers { permanent_uuid: "6192b6938bfd4cb49a7f1f5b0e13106c" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33541 } } peers { permanent_uuid: "0b1b8ac5edc54a20a4ea3751e8fa26ca" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35615 } }
I20260808 06:08:37.695624 30174 heartbeater.cc:503] Master 127.1.151.190:33715 was elected leader, sending a full tablet report...
I20260808 06:08:37.695533 30408 raft_consensus.cc:348] T e068ff017692408f8a3d32df2d85bb01 P 0b1b8ac5edc54a20a4ea3751e8fa26ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc901283a4e4ae593572fc97ab181d2" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44555 } } peers { permanent_uuid: "6192b6938bfd4cb49a7f1f5b0e13106c" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33541 } } peers { permanent_uuid: "0b1b8ac5edc54a20a4ea3751e8fa26ca" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35615 } }
I20260808 06:08:37.695533 30409 raft_consensus.cc:348] T e068ff017692408f8a3d32df2d85bb01 P 6192b6938bfd4cb49a7f1f5b0e13106c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc901283a4e4ae593572fc97ab181d2" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44555 } } peers { permanent_uuid: "6192b6938bfd4cb49a7f1f5b0e13106c" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33541 } } peers { permanent_uuid: "0b1b8ac5edc54a20a4ea3751e8fa26ca" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35615 } }
I20260808 06:08:37.695784 30409 raft_consensus.cc:374] T e068ff017692408f8a3d32df2d85bb01 P 6192b6938bfd4cb49a7f1f5b0e13106c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:08:37.695784 30408 raft_consensus.cc:374] T e068ff017692408f8a3d32df2d85bb01 P 0b1b8ac5edc54a20a4ea3751e8fa26ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:08:37.695842 30408 raft_consensus.cc:729] T e068ff017692408f8a3d32df2d85bb01 P 0b1b8ac5edc54a20a4ea3751e8fa26ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b1b8ac5edc54a20a4ea3751e8fa26ca, State: Initialized, Role: FOLLOWER
I20260808 06:08:37.695842 30409 raft_consensus.cc:729] T e068ff017692408f8a3d32df2d85bb01 P 6192b6938bfd4cb49a7f1f5b0e13106c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6192b6938bfd4cb49a7f1f5b0e13106c, State: Initialized, Role: FOLLOWER
I20260808 06:08:37.696275 30408 consensus_queue.cc:260] T e068ff017692408f8a3d32df2d85bb01 P 0b1b8ac5edc54a20a4ea3751e8fa26ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc901283a4e4ae593572fc97ab181d2" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44555 } } peers { permanent_uuid: "6192b6938bfd4cb49a7f1f5b0e13106c" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33541 } } peers { permanent_uuid: "0b1b8ac5edc54a20a4ea3751e8fa26ca" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35615 } }
I20260808 06:08:37.696275 30409 consensus_queue.cc:260] T e068ff017692408f8a3d32df2d85bb01 P 6192b6938bfd4cb49a7f1f5b0e13106c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc901283a4e4ae593572fc97ab181d2" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44555 } } peers { permanent_uuid: "6192b6938bfd4cb49a7f1f5b0e13106c" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33541 } } peers { permanent_uuid: "0b1b8ac5edc54a20a4ea3751e8fa26ca" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35615 } }
I20260808 06:08:37.697307 30409 ts_tablet_manager.cc:1428] T e068ff017692408f8a3d32df2d85bb01 P 6192b6938bfd4cb49a7f1f5b0e13106c: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 06:08:37.697307 30408 ts_tablet_manager.cc:1428] T e068ff017692408f8a3d32df2d85bb01 P 0b1b8ac5edc54a20a4ea3751e8fa26ca: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 06:08:37.697583 30407 ts_tablet_manager.cc:1428] T e068ff017692408f8a3d32df2d85bb01 P 2cc901283a4e4ae593572fc97ab181d2: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260808 06:08:37.697659 30379 heartbeater.cc:503] Master 127.1.151.190:33715 was elected leader, sending a full tablet report...
I20260808 06:08:37.697963 29974 heartbeater.cc:503] Master 127.1.151.190:33715 was elected leader, sending a full tablet report...
I20260808 06:08:37.704192 30414 raft_consensus.cc:482] T e068ff017692408f8a3d32df2d85bb01 P 2cc901283a4e4ae593572fc97ab181d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:08:37.704350 30414 raft_consensus.cc:504] T e068ff017692408f8a3d32df2d85bb01 P 2cc901283a4e4ae593572fc97ab181d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc901283a4e4ae593572fc97ab181d2" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44555 } } peers { permanent_uuid: "6192b6938bfd4cb49a7f1f5b0e13106c" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33541 } } peers { permanent_uuid: "0b1b8ac5edc54a20a4ea3751e8fa26ca" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35615 } }
I20260808 06:08:37.705768 30414 leader_election.cc:288] T e068ff017692408f8a3d32df2d85bb01 P 2cc901283a4e4ae593572fc97ab181d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6192b6938bfd4cb49a7f1f5b0e13106c (127.1.151.131:33541), 0b1b8ac5edc54a20a4ea3751e8fa26ca (127.1.151.129:35615)
I20260808 06:08:37.709465 30402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:37.705862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:57473 (local address 127.1.151.131:33541)
0808 06:08:37.705971 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:37.705975 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:37.705988 (+ 13us) server_negotiation.cc:407] Connection header received
0808 06:08:37.706128 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:37.706131 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:37.706215 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:37.706300 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:08:37.706604 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:37.707256 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.708420 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:37.708907 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.709003 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:37.709047 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:37.709264 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 06:08:37.709334 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 06:08:37.709466 30177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:37.705738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:33541 (local address 127.1.151.130:57473)
0808 06:08:37.705876 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:37.705892 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:37.705981 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:37.706330 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:37.706333 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:37.706354 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:08:37.706490 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:37.706498 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:37.707368 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.707372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:37.708298 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:37.708309 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:37.709041 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.709045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:37.709148 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:37.709175 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 06:08:37.709244 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 06:08:37.709328 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":22}
I20260808 06:08:37.709740 30418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:37.705744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:35615 (local address 127.1.151.130:35105)
0808 06:08:37.706330 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:37.706342 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:37.706433 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:37.706695 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:37.706698 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:37.706717 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:08:37.706838 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:37.706844 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:37.707681 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.707684 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:37.708558 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:37.708569 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:37.709294 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.709297 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:37.709401 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:37.709424 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 06:08:37.709517 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 06:08:37.709583 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"mutex_wait_us":205,"thread_start_us":100,"threads_started":1}
I20260808 06:08:37.709740 30406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:37.706119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:35105 (local address 127.1.151.129:35615)
0808 06:08:37.706249 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:37.706252 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:37.706416 (+ 164us) server_negotiation.cc:407] Connection header received
0808 06:08:37.706508 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:37.706511 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:37.706570 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:37.706637 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:08:37.706957 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:37.707515 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.708704 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:37.709157 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.709282 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:37.709330 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:37.709551 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 06:08:37.709626 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 06:08:37.710217 30309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e068ff017692408f8a3d32df2d85bb01" candidate_uuid: "2cc901283a4e4ae593572fc97ab181d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6192b6938bfd4cb49a7f1f5b0e13106c" is_pre_election: true
I20260808 06:08:37.710425 29894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e068ff017692408f8a3d32df2d85bb01" candidate_uuid: "2cc901283a4e4ae593572fc97ab181d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1b8ac5edc54a20a4ea3751e8fa26ca" is_pre_election: true
I20260808 06:08:37.710496 30309 raft_consensus.cc:2454] T e068ff017692408f8a3d32df2d85bb01 P 6192b6938bfd4cb49a7f1f5b0e13106c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cc901283a4e4ae593572fc97ab181d2 in term 0.
I20260808 06:08:37.710682 29894 raft_consensus.cc:2454] T e068ff017692408f8a3d32df2d85bb01 P 0b1b8ac5edc54a20a4ea3751e8fa26ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cc901283a4e4ae593572fc97ab181d2 in term 0.
I20260808 06:08:37.711027 30000 leader_election.cc:302] T e068ff017692408f8a3d32df2d85bb01 P 2cc901283a4e4ae593572fc97ab181d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0b1b8ac5edc54a20a4ea3751e8fa26ca, 2cc901283a4e4ae593572fc97ab181d2, 6192b6938bfd4cb49a7f1f5b0e13106c; no voters:
I20260808 06:08:37.711287 30414 raft_consensus.cc:2781] T e068ff017692408f8a3d32df2d85bb01 P 2cc901283a4e4ae593572fc97ab181d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 06:08:37.711364 30414 raft_consensus.cc:482] T e068ff017692408f8a3d32df2d85bb01 P 2cc901283a4e4ae593572fc97ab181d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 06:08:37.711393 30414 raft_consensus.cc:3037] T e068ff017692408f8a3d32df2d85bb01 P 2cc901283a4e4ae593572fc97ab181d2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:08:37.712422 30414 raft_consensus.cc:504] T e068ff017692408f8a3d32df2d85bb01 P 2cc901283a4e4ae593572fc97ab181d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc901283a4e4ae593572fc97ab181d2" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44555 } } peers { permanent_uuid: "6192b6938bfd4cb49a7f1f5b0e13106c" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33541 } } peers { permanent_uuid: "0b1b8ac5edc54a20a4ea3751e8fa26ca" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35615 } }
I20260808 06:08:37.712756 30414 leader_election.cc:288] T e068ff017692408f8a3d32df2d85bb01 P 2cc901283a4e4ae593572fc97ab181d2 [CANDIDATE]: Term 1 election: Requested vote from peers 6192b6938bfd4cb49a7f1f5b0e13106c (127.1.151.131:33541), 0b1b8ac5edc54a20a4ea3751e8fa26ca (127.1.151.129:35615)
I20260808 06:08:37.713124 30309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e068ff017692408f8a3d32df2d85bb01" candidate_uuid: "2cc901283a4e4ae593572fc97ab181d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6192b6938bfd4cb49a7f1f5b0e13106c"
I20260808 06:08:37.713233 30309 raft_consensus.cc:3037] T e068ff017692408f8a3d32df2d85bb01 P 6192b6938bfd4cb49a7f1f5b0e13106c [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:08:37.713239 29894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e068ff017692408f8a3d32df2d85bb01" candidate_uuid: "2cc901283a4e4ae593572fc97ab181d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1b8ac5edc54a20a4ea3751e8fa26ca"
I20260808 06:08:37.713353 29894 raft_consensus.cc:3037] T e068ff017692408f8a3d32df2d85bb01 P 0b1b8ac5edc54a20a4ea3751e8fa26ca [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:08:37.714414 30309 raft_consensus.cc:2454] T e068ff017692408f8a3d32df2d85bb01 P 6192b6938bfd4cb49a7f1f5b0e13106c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cc901283a4e4ae593572fc97ab181d2 in term 1.
I20260808 06:08:37.714429 29894 raft_consensus.cc:2454] T e068ff017692408f8a3d32df2d85bb01 P 0b1b8ac5edc54a20a4ea3751e8fa26ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cc901283a4e4ae593572fc97ab181d2 in term 1.
I20260808 06:08:37.714736 30000 leader_election.cc:302] T e068ff017692408f8a3d32df2d85bb01 P 2cc901283a4e4ae593572fc97ab181d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc901283a4e4ae593572fc97ab181d2, 6192b6938bfd4cb49a7f1f5b0e13106c; no voters:
I20260808 06:08:37.714915 30414 raft_consensus.cc:2781] T e068ff017692408f8a3d32df2d85bb01 P 2cc901283a4e4ae593572fc97ab181d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:08:37.715155 30414 raft_consensus.cc:686] T e068ff017692408f8a3d32df2d85bb01 P 2cc901283a4e4ae593572fc97ab181d2 [term 1 LEADER]: Becoming Leader. State: Replica: 2cc901283a4e4ae593572fc97ab181d2, State: Running, Role: LEADER
I20260808 06:08:37.715461 30414 consensus_queue.cc:237] T e068ff017692408f8a3d32df2d85bb01 P 2cc901283a4e4ae593572fc97ab181d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc901283a4e4ae593572fc97ab181d2" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44555 } } peers { permanent_uuid: "6192b6938bfd4cb49a7f1f5b0e13106c" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33541 } } peers { permanent_uuid: "0b1b8ac5edc54a20a4ea3751e8fa26ca" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35615 } }
I20260808 06:08:37.718804 29668 catalog_manager.cc:5495] T e068ff017692408f8a3d32df2d85bb01 P 2cc901283a4e4ae593572fc97ab181d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2cc901283a4e4ae593572fc97ab181d2 (127.1.151.130). New cstate: current_term: 1 leader_uuid: "2cc901283a4e4ae593572fc97ab181d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cc901283a4e4ae593572fc97ab181d2" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 44555 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6192b6938bfd4cb49a7f1f5b0e13106c" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b1b8ac5edc54a20a4ea3751e8fa26ca" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35615 } health_report { overall_health: UNKNOWN } } }
I20260808 06:08:37.736511 30400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:37.733281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47544 (local address 127.1.151.130:44555)
0808 06:08:37.733397 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:37.733401 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:37.733414 (+ 13us) server_negotiation.cc:407] Connection header received
0808 06:08:37.733491 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:37.733493 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:37.733548 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:37.733641 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:08:37.733854 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:37.734409 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.735255 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:37.735469 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:37.735544 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:37.735807 (+ 263us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:08:37.736224 (+ 417us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:08:37.736309 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:37.736337 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 06:08:37.736382 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 06:08:37.742609 29894 raft_consensus.cc:1261] T e068ff017692408f8a3d32df2d85bb01 P 0b1b8ac5edc54a20a4ea3751e8fa26ca [term 1 FOLLOWER]: Refusing update from remote peer 2cc901283a4e4ae593572fc97ab181d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:08:37.742609 30309 raft_consensus.cc:1261] T e068ff017692408f8a3d32df2d85bb01 P 6192b6938bfd4cb49a7f1f5b0e13106c [term 1 FOLLOWER]: Refusing update from remote peer 2cc901283a4e4ae593572fc97ab181d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:08:37.743613 30419 consensus_queue.cc:1035] T e068ff017692408f8a3d32df2d85bb01 P 2cc901283a4e4ae593572fc97ab181d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b1b8ac5edc54a20a4ea3751e8fa26ca" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 35615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:08:37.743810 30414 consensus_queue.cc:1035] T e068ff017692408f8a3d32df2d85bb01 P 2cc901283a4e4ae593572fc97ab181d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6192b6938bfd4cb49a7f1f5b0e13106c" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 33541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 06:08:37.751420 29975 tablet.cc:2369] T e068ff017692408f8a3d32df2d85bb01 P 0b1b8ac5edc54a20a4ea3751e8fa26ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 06:08:37.751708 30423 mvcc.cc:205] Tried to move back new op lower bound from 7316149525465821184 to 7316149525365432320. Current Snapshot: MvccSnapshot[applied={T|T < 7316149525465821184}]
I20260808 06:08:37.753355 30424 mvcc.cc:205] Tried to move back new op lower bound from 7316149525465821184 to 7316149525365432320. Current Snapshot: MvccSnapshot[applied={T|T < 7316149525465821184}]
I20260808 06:08:37.761030 29668 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40978:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:08:37.761214 29668 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40978:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:08:37.763767 29668 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5586251030f940e6b4b4243d1c811c03: Sending DeleteTablet for 3 replicas of tablet e068ff017692408f8a3d32df2d85bb01
I20260808 06:08:37.764564 29858 tablet_service.cc:1514] Processing DeleteTablet for tablet e068ff017692408f8a3d32df2d85bb01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:08:37 UTC) from {username='slave'} at 127.0.0.1:46556
I20260808 06:08:37.764612 30281 tablet_service.cc:1514] Processing DeleteTablet for tablet e068ff017692408f8a3d32df2d85bb01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:08:37 UTC) from {username='slave'} at 127.0.0.1:43056
I20260808 06:08:37.764720 30066 tablet_service.cc:1514] Processing DeleteTablet for tablet e068ff017692408f8a3d32df2d85bb01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:08:37 UTC) from {username='slave'} at 127.0.0.1:47528
I20260808 06:08:37.765120 30436 tablet_replica.cc:330] T e068ff017692408f8a3d32df2d85bb01 P 0b1b8ac5edc54a20a4ea3751e8fa26ca: stopping tablet replica
I20260808 06:08:37.765200 30437 tablet_replica.cc:330] T e068ff017692408f8a3d32df2d85bb01 P 6192b6938bfd4cb49a7f1f5b0e13106c: stopping tablet replica
I20260808 06:08:37.765390 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 29759
I20260808 06:08:37.765431 30439 tablet_replica.cc:330] T e068ff017692408f8a3d32df2d85bb01 P 2cc901283a4e4ae593572fc97ab181d2: stopping tablet replica
I20260808 06:08:37.777565 30439 raft_consensus.cc:2229] T e068ff017692408f8a3d32df2d85bb01 P 2cc901283a4e4ae593572fc97ab181d2 [term 1 LEADER]: Raft consensus shutting down.
I20260808 06:08:37.778036 30439 raft_consensus.cc:2258] T e068ff017692408f8a3d32df2d85bb01 P 2cc901283a4e4ae593572fc97ab181d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:08:37.778553 30437 raft_consensus.cc:2229] T e068ff017692408f8a3d32df2d85bb01 P 6192b6938bfd4cb49a7f1f5b0e13106c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:08:37.778857 30437 raft_consensus.cc:2258] T e068ff017692408f8a3d32df2d85bb01 P 6192b6938bfd4cb49a7f1f5b0e13106c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:08:37.780223 30439 ts_tablet_manager.cc:1905] T e068ff017692408f8a3d32df2d85bb01 P 2cc901283a4e4ae593572fc97ab181d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:08:37.782629 30439 ts_tablet_manager.cc:1918] T e068ff017692408f8a3d32df2d85bb01 P 2cc901283a4e4ae593572fc97ab181d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:08:37.782768 30439 log.cc:1196] T e068ff017692408f8a3d32df2d85bb01 P 2cc901283a4e4ae593572fc97ab181d2: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-1/wal/wals/e068ff017692408f8a3d32df2d85bb01
I20260808 06:08:37.783115 30439 ts_tablet_manager.cc:1939] T e068ff017692408f8a3d32df2d85bb01 P 2cc901283a4e4ae593572fc97ab181d2: Deleting consensus metadata
I20260808 06:08:37.783740 30437 ts_tablet_manager.cc:1905] T e068ff017692408f8a3d32df2d85bb01 P 6192b6938bfd4cb49a7f1f5b0e13106c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:08:37.784507 29646 catalog_manager.cc:4841] TS 2cc901283a4e4ae593572fc97ab181d2 (127.1.151.130:44555): tablet e068ff017692408f8a3d32df2d85bb01 (table test-table [id=80ac5ca83d14452fb4c65569fb9353d0]) successfully deleted
W20260808 06:08:37.785069 29646 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.151.129:35615 (error 108)
I20260808 06:08:37.785310 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 29977
I20260808 06:08:37.787463 30437 ts_tablet_manager.cc:1918] T e068ff017692408f8a3d32df2d85bb01 P 6192b6938bfd4cb49a7f1f5b0e13106c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:08:37.787591 30437 log.cc:1196] T e068ff017692408f8a3d32df2d85bb01 P 6192b6938bfd4cb49a7f1f5b0e13106c: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786169070948234-1630-0/minicluster-data/ts-2/wal/wals/e068ff017692408f8a3d32df2d85bb01
I20260808 06:08:37.787921 30437 ts_tablet_manager.cc:1939] T e068ff017692408f8a3d32df2d85bb01 P 6192b6938bfd4cb49a7f1f5b0e13106c: Deleting consensus metadata
I20260808 06:08:37.788625 30397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:37.788288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:35615 (local address 127.0.0.1:46562)
0808 06:08:37.788437 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:37.788538 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.129:35615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 06:08:37.790305 29645 catalog_manager.cc:4841] TS 6192b6938bfd4cb49a7f1f5b0e13106c (127.1.151.131:33541): tablet e068ff017692408f8a3d32df2d85bb01 (table test-table [id=80ac5ca83d14452fb4c65569fb9353d0]) successfully deleted
W20260808 06:08:37.790371 29646 catalog_manager.cc:4584] TS 0b1b8ac5edc54a20a4ea3751e8fa26ca (127.1.151.129:35615): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e068ff017692408f8a3d32df2d85bb01: Network error: Client connection negotiation failed: client connection to 127.1.151.129:35615: connect: Connection refused (error 111)
I20260808 06:08:37.795704 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 30179
I20260808 06:08:37.808852 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 29609
2026-08-08T06:08:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1013 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:08:37 dist-test-slave-zj8q krb5kdc[30446](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:08:37 dist-test-slave-zj8q krb5kdc[30446](info): set up 2 sockets
Aug 08 06:08:37 dist-test-slave-zj8q krb5kdc[30446](info): commencing operation
krb5kdc: starting...
W20260808 06:08:38.953081 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:08:39 dist-test-slave-zj8q krb5kdc[30446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:08:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:08:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.151.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
I20260808 06:08:39.072618 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:34155
--webserver_interface=127.1.151.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:43065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:34155
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:08:39.181737 30466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:39.181982 30466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:39.182021 30466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:39.185866 30466 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:08:39.185917 30466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:39.185935 30466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:39.185948 30466 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:08:39.185962 30466 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:08:39.190928 30466 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:43065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:34155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:34155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:39.191921 30466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:39.193019 30466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:08:39.201118 30480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:39.201159 30483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:39.201144 30481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:39.201274 30466 server_base.cc:1034] running on GCE node
I20260808 06:08:39.201913 30466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:39.202986 30466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:39.204197 30466 hybrid_clock.cc:648] HybridClock initialized: now 1786169319204172 us; error 40 us; skew 500 ppm
Aug 08 06:08:39 dist-test-slave-zj8q krb5kdc[30446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169319, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:08:39.206914 30466 init.cc:377] Logged in from keytab as kudu/127.1.151.190@KRBTEST.COM (short username kudu)
I20260808 06:08:39.207775 30466 webserver.cc:460] Webserver started at http://127.1.151.190:40739/ using document root <none> and password file <none>
I20260808 06:08:39.208300 30466 fs_manager.cc:359] Metadata directory not provided
I20260808 06:08:39.208366 30466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:08:39.208563 30466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:08:39.210631 30466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "8eaa3d2db5c646f1b25e541afd445f34"
format_stamp: "Formatted at 2026-08-08 06:08:39 on dist-test-slave-zj8q"
server_key: "053b032f6b49e599694631b8a0d41f8e"
server_key_iv: "af593b31ac2c68aee73f31bb1468d032"
server_key_version: "encryptionkey@0"
I20260808 06:08:39.211166 30466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "8eaa3d2db5c646f1b25e541afd445f34"
format_stamp: "Formatted at 2026-08-08 06:08:39 on dist-test-slave-zj8q"
server_key: "053b032f6b49e599694631b8a0d41f8e"
server_key_iv: "af593b31ac2c68aee73f31bb1468d032"
server_key_version: "encryptionkey@0"
I20260808 06:08:39.214712 30466 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 06:08:39.217036 30493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:39.218319 30466 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:08:39.218421 30466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "8eaa3d2db5c646f1b25e541afd445f34"
format_stamp: "Formatted at 2026-08-08 06:08:39 on dist-test-slave-zj8q"
server_key: "053b032f6b49e599694631b8a0d41f8e"
server_key_iv: "af593b31ac2c68aee73f31bb1468d032"
server_key_version: "encryptionkey@0"
I20260808 06:08:39.218506 30466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:08:39.234916 30466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:08:39.237819 30466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:08:39.237962 30466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:08:39.244681 30466 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:34155
I20260808 06:08:39.244725 30567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:34155 every 8 connection(s)
I20260808 06:08:39.245648 30466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:08:39.248070 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 30466
I20260808 06:08:39.248183 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:08:39.248459 1630 external_mini_cluster.cc:1428] Setting key 2f1129054163cfb3436c1b928afe35a4
I20260808 06:08:39.248652 30568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:08:39.255858 30568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8eaa3d2db5c646f1b25e541afd445f34: Bootstrap starting.
Aug 08 06:08:39 dist-test-slave-zj8q krb5kdc[30446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:08:39.258252 30568 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8eaa3d2db5c646f1b25e541afd445f34: Neither blocks nor log segments found. Creating new log.
I20260808 06:08:39.259057 30568 log.cc:824] T 00000000000000000000000000000000 P 8eaa3d2db5c646f1b25e541afd445f34: Log is configured to *not* fsync() on all Append() calls
I20260808 06:08:39.261003 30568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8eaa3d2db5c646f1b25e541afd445f34: No bootstrap required, opened a new log
I20260808 06:08:39.262989 30571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:39.249885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41966 (local address 127.1.151.190:34155)
0808 06:08:39.250345 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:39.250363 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:39.250405 (+ 42us) server_negotiation.cc:407] Connection header received
0808 06:08:39.251375 (+ 970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:39.251402 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:39.251822 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:39.252181 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:39.252870 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:39.254404 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:39.255289 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:39.255971 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:39.256120 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:39.258467 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:39.258472 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:39.258486 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:08:39.258514 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:39.260260 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:39.260674 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:39.260680 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:39.260684 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:39.260794 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:39.261057 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:39.261060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:39.261062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:39.261478 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:39.261669 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:39.261962 (+ 293us) server_negotiation.cc:299] Negotiation successful
0808 06:08:39.262283 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":132,"threads_started":1}
I20260808 06:08:39.264516 30568 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8eaa3d2db5c646f1b25e541afd445f34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eaa3d2db5c646f1b25e541afd445f34" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34155 } }
I20260808 06:08:39.264737 30568 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8eaa3d2db5c646f1b25e541afd445f34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:08:39.264796 30568 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8eaa3d2db5c646f1b25e541afd445f34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eaa3d2db5c646f1b25e541afd445f34, State: Initialized, Role: FOLLOWER
I20260808 06:08:39.265235 30568 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8eaa3d2db5c646f1b25e541afd445f34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eaa3d2db5c646f1b25e541afd445f34" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34155 } }
I20260808 06:08:39.265362 30568 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8eaa3d2db5c646f1b25e541afd445f34 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:08:39.265406 30568 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8eaa3d2db5c646f1b25e541afd445f34 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:08:39.265475 30568 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8eaa3d2db5c646f1b25e541afd445f34 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:08:39.266460 30568 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8eaa3d2db5c646f1b25e541afd445f34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eaa3d2db5c646f1b25e541afd445f34" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34155 } }
I20260808 06:08:39.266773 30568 leader_election.cc:302] T 00000000000000000000000000000000 P 8eaa3d2db5c646f1b25e541afd445f34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8eaa3d2db5c646f1b25e541afd445f34; no voters:
I20260808 06:08:39.267112 30568 leader_election.cc:288] T 00000000000000000000000000000000 P 8eaa3d2db5c646f1b25e541afd445f34 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:08:39.267204 30578 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8eaa3d2db5c646f1b25e541afd445f34 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:08:39.267409 30578 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8eaa3d2db5c646f1b25e541afd445f34 [term 1 LEADER]: Becoming Leader. State: Replica: 8eaa3d2db5c646f1b25e541afd445f34, State: Running, Role: LEADER
I20260808 06:08:39.267729 30578 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8eaa3d2db5c646f1b25e541afd445f34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eaa3d2db5c646f1b25e541afd445f34" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34155 } }
I20260808 06:08:39.268213 30568 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8eaa3d2db5c646f1b25e541afd445f34 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:08:39.269240 30579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8eaa3d2db5c646f1b25e541afd445f34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8eaa3d2db5c646f1b25e541afd445f34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eaa3d2db5c646f1b25e541afd445f34" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34155 } } }
I20260808 06:08:39.269381 30579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8eaa3d2db5c646f1b25e541afd445f34 [sys.catalog]: This master's current role is: LEADER
I20260808 06:08:39.269742 30581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8eaa3d2db5c646f1b25e541afd445f34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8eaa3d2db5c646f1b25e541afd445f34. Latest consensus state: current_term: 1 leader_uuid: "8eaa3d2db5c646f1b25e541afd445f34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eaa3d2db5c646f1b25e541afd445f34" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34155 } } }
I20260808 06:08:39.270390 30581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8eaa3d2db5c646f1b25e541afd445f34 [sys.catalog]: This master's current role is: LEADER
I20260808 06:08:39.270509 30588 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:08:39.273576 30588 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:08:39.278859 30588 catalog_manager.cc:1347] Generated new cluster ID: a788c539f2784576987be0aa6c18d2db
I20260808 06:08:39.278924 30588 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:08:39.293416 30588 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:08:39.294378 30588 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:08:39.305258 30588 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8eaa3d2db5c646f1b25e541afd445f34: Generated new TSK 0
I20260808 06:08:39.305981 30588 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.151.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab.
Entry for principal kudu/127.1.151.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab.
I20260808 06:08:39.339053 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.129:0
--local_ip_for_outbound_sockets=127.1.151.129
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:34155
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:43065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:08:39.459604 30608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:39.459842 30608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:39.459882 30608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:39.463744 30608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:39.463790 30608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:39.463843 30608 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.129
I20260808 06:08:39.468873 30608 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:43065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:34155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.129
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:39.469760 30608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:39.470738 30608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:08:39.480420 30615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:39.480577 30617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:39.480723 30619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:39.480912 30608 server_base.cc:1034] running on GCE node
I20260808 06:08:39.481462 30608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:39.482242 30608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:39.485561 30608 hybrid_clock.cc:648] HybridClock initialized: now 1786169319485525 us; error 43 us; skew 500 ppm
Aug 08 06:08:39 dist-test-slave-zj8q krb5kdc[30446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169319, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:08:39.492017 30608 init.cc:377] Logged in from keytab as kudu/127.1.151.129@KRBTEST.COM (short username kudu)
I20260808 06:08:39.492811 30608 webserver.cc:460] Webserver started at http://127.1.151.129:44405/ using document root <none> and password file <none>
I20260808 06:08:39.493335 30608 fs_manager.cc:359] Metadata directory not provided
I20260808 06:08:39.493393 30608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:08:39.493556 30608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:08:39.495577 30608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-0/data/instance:
uuid: "e1a3c69a09a740ee8d6a10c32404d922"
format_stamp: "Formatted at 2026-08-08 06:08:39 on dist-test-slave-zj8q"
server_key: "30cf9d8ed855593494c62a93a59f5215"
server_key_iv: "8894a05d46ce49fe8c8b7a912b5184a2"
server_key_version: "encryptionkey@0"
I20260808 06:08:39.496096 30608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance:
uuid: "e1a3c69a09a740ee8d6a10c32404d922"
format_stamp: "Formatted at 2026-08-08 06:08:39 on dist-test-slave-zj8q"
server_key: "30cf9d8ed855593494c62a93a59f5215"
server_key_iv: "8894a05d46ce49fe8c8b7a912b5184a2"
server_key_version: "encryptionkey@0"
I20260808 06:08:39.499610 30608 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 06:08:39.501883 30635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:39.503051 30608 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:08:39.503154 30608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-0/wal
uuid: "e1a3c69a09a740ee8d6a10c32404d922"
format_stamp: "Formatted at 2026-08-08 06:08:39 on dist-test-slave-zj8q"
server_key: "30cf9d8ed855593494c62a93a59f5215"
server_key_iv: "8894a05d46ce49fe8c8b7a912b5184a2"
server_key_version: "encryptionkey@0"
I20260808 06:08:39.503257 30608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:08:39.521026 30608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:08:39.523806 30608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:08:39.523964 30608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:08:39.524572 30608 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:08:39.525465 30608 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:08:39.525514 30608 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:39.525574 30608 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:08:39.525604 30608 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:39.533908 30608 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.129:41567
I20260808 06:08:39.533936 30803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.129:41567 every 8 connection(s)
I20260808 06:08:39.534781 30608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
I20260808 06:08:39.535848 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 30608
I20260808 06:08:39.535983 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance
I20260808 06:08:39.536291 1630 external_mini_cluster.cc:1428] Setting key 1ae5b7a4f27f731ebeec00b98fb5783f
Aug 08 06:08:39 dist-test-slave-zj8q krb5kdc[30446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169319, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.151.130@KRBTEST.COM; defaulting to no policy
I20260808 06:08:39.548925 30571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:39.536573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:60153 (local address 127.1.151.190:34155)
0808 06:08:39.536727 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:39.536732 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:39.537678 (+ 946us) server_negotiation.cc:407] Connection header received
0808 06:08:39.538748 (+ 1070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:39.538751 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:39.538795 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:39.538862 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:39.540306 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:39.541116 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:39.542251 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:39.542583 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:39.542706 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:39.545214 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:39.545218 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:39.545222 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:08:39.545252 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:39.546939 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:39.547423 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:39.547426 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:39.547428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:39.547528 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:39.547850 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:39.547853 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:39.547854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:39.548038 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:39.548116 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:39.548679 (+ 563us) server_negotiation.cc:299] Negotiation successful
0808 06:08:39.548795 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 06:08:39.549768 30806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:39.537029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34155 (local address 127.1.151.129:60153)
0808 06:08:39.537509 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:39.537548 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:39.538533 (+ 985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:39.539035 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:39.539044 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:39.539501 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:08:39.540131 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:39.540153 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:39.541238 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:39.541241 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:39.542116 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:39.542125 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:39.542731 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:39.542734 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:39.542872 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:39.543415 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:08:39.543431 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:08:39.545038 (+ 1607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:08:39.547054 (+ 2016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:08:39.547059 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:08:39.547069 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:08:39.547324 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:08:39.547624 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:08:39.547627 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:08:39.547629 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:08:39.547762 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:08:39.548159 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:08:39.548164 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:08:39.548390 (+ 226us) client_negotiation.cc:769] Sending connection context
0808 06:08:39.548601 (+ 211us) client_negotiation.cc:240] Negotiation successful
0808 06:08:39.548895 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":143,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.130@KRBTEST.COM" created.
I20260808 06:08:39.550928 30804 heartbeater.cc:347] Connected to a master server at 127.1.151.190:34155
I20260808 06:08:39.551189 30804 heartbeater.cc:464] Registering TS with master...
I20260808 06:08:39.551723 30804 heartbeater.cc:511] Master 127.1.151.190:34155 requested a full tablet report, sending...
I20260808 06:08:39.553493 30517 ts_manager.cc:194] Registered new tserver with Master: e1a3c69a09a740ee8d6a10c32404d922 (127.1.151.129:41567)
I20260808 06:08:39.555119 30517 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.129@KRBTEST.COM'} at 127.1.151.129:60153
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab.
Entry for principal kudu/127.1.151.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab.
I20260808 06:08:39.565104 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.130:0
--local_ip_for_outbound_sockets=127.1.151.130
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:34155
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:43065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:08:39.674322 30810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:39.674562 30810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:39.674603 30810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:39.678505 30810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:39.678553 30810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:39.678607 30810 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.130
I20260808 06:08:39.683591 30810 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:43065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:34155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30810
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.130
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:39.684540 30810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:39.685496 30810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:08:39.694903 30816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:39.695053 30810 server_base.cc:1034] running on GCE node
W20260808 06:08:39.694924 30821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:39.695194 30818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:39.695555 30810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:39.696177 30810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:39.697355 30810 hybrid_clock.cc:648] HybridClock initialized: now 1786169319697329 us; error 34 us; skew 500 ppm
Aug 08 06:08:39 dist-test-slave-zj8q krb5kdc[30446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169319, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:08:39.700062 30810 init.cc:377] Logged in from keytab as kudu/127.1.151.130@KRBTEST.COM (short username kudu)
I20260808 06:08:39.700901 30810 webserver.cc:460] Webserver started at http://127.1.151.130:41727/ using document root <none> and password file <none>
I20260808 06:08:39.701421 30810 fs_manager.cc:359] Metadata directory not provided
I20260808 06:08:39.701478 30810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:08:39.701645 30810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:08:39.703652 30810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-1/data/instance:
uuid: "9e583c670f1747f9ad2aafe0bbf0bc3c"
format_stamp: "Formatted at 2026-08-08 06:08:39 on dist-test-slave-zj8q"
server_key: "b3f1894412f1c2de981d6e489060f156"
server_key_iv: "7ce929d0735a344f8fbb3fa11904ae78"
server_key_version: "encryptionkey@0"
I20260808 06:08:39.704265 30810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance:
uuid: "9e583c670f1747f9ad2aafe0bbf0bc3c"
format_stamp: "Formatted at 2026-08-08 06:08:39 on dist-test-slave-zj8q"
server_key: "b3f1894412f1c2de981d6e489060f156"
server_key_iv: "7ce929d0735a344f8fbb3fa11904ae78"
server_key_version: "encryptionkey@0"
I20260808 06:08:39.708089 30810 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 06:08:39.710472 30831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:39.711652 30810 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 06:08:39.711764 30810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-1/wal
uuid: "9e583c670f1747f9ad2aafe0bbf0bc3c"
format_stamp: "Formatted at 2026-08-08 06:08:39 on dist-test-slave-zj8q"
server_key: "b3f1894412f1c2de981d6e489060f156"
server_key_iv: "7ce929d0735a344f8fbb3fa11904ae78"
server_key_version: "encryptionkey@0"
I20260808 06:08:39.711853 30810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:08:39.733732 30810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:08:39.736621 30810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:08:39.736832 30810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:08:39.737404 30810 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:08:39.738255 30810 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:08:39.738304 30810 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:39.738348 30810 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:08:39.738376 30810 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:39.746527 30810 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.130:39773
I20260808 06:08:39.746613 31004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.130:39773 every 8 connection(s)
I20260808 06:08:39.747406 30810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
I20260808 06:08:39.751657 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 30810
I20260808 06:08:39.751799 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance
I20260808 06:08:39.752162 1630 external_mini_cluster.cc:1428] Setting key 99dba36e38dbe8f4b2374462ba4adb7c
Aug 08 06:08:39 dist-test-slave-zj8q krb5kdc[30446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169319, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:08:39.761770 30571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:39.749193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:35013 (local address 127.1.151.190:34155)
0808 06:08:39.749334 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:39.749339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:39.750195 (+ 856us) server_negotiation.cc:407] Connection header received
0808 06:08:39.751488 (+ 1293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:39.751491 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:39.751538 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:39.751609 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:39.753087 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:39.753777 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:39.755328 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:39.755545 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:39.755620 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:39.758209 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:39.758213 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:39.758216 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:08:39.758244 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:39.759730 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:39.760315 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:39.760318 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:39.760319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:39.760393 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:39.760753 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:39.760755 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:39.760756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:39.760922 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:39.760993 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:39.761541 (+ 548us) server_negotiation.cc:299] Negotiation successful
0808 06:08:39.761650 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 06:08:39.762866 31013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:39.749563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34155 (local address 127.1.151.130:35013)
0808 06:08:39.750025 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:39.750067 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:39.751255 (+ 1188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:39.751830 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:39.751838 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:39.752330 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:08:39.752930 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:39.752950 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:39.753906 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:39.753910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:39.755219 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:39.755231 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:39.755681 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:39.755685 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:39.755890 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:39.756451 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:08:39.756468 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:08:39.758016 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:08:39.759861 (+ 1845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:08:39.759867 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:08:39.759883 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:08:39.760224 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:08:39.760475 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:08:39.760478 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:08:39.760480 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:08:39.760669 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:08:39.761014 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:08:39.761021 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:08:39.761319 (+ 298us) client_negotiation.cc:769] Sending connection context
0808 06:08:39.761521 (+ 202us) client_negotiation.cc:240] Negotiation successful
0808 06:08:39.761856 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":116,"threads_started":1}
WARNING: no policy specified for kudu/127.1.151.131@KRBTEST.COM; defaulting to no policy
I20260808 06:08:39.763995 31007 heartbeater.cc:347] Connected to a master server at 127.1.151.190:34155
I20260808 06:08:39.764225 31007 heartbeater.cc:464] Registering TS with master...
I20260808 06:08:39.764699 31007 heartbeater.cc:511] Master 127.1.151.190:34155 requested a full tablet report, sending...
I20260808 06:08:39.765779 30517 ts_manager.cc:194] Registered new tserver with Master: 9e583c670f1747f9ad2aafe0bbf0bc3c (127.1.151.130:39773)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.131@KRBTEST.COM" created.
I20260808 06:08:39.766407 30517 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.130@KRBTEST.COM'} at 127.1.151.130:35013
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab.
Entry for principal kudu/127.1.151.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab.
I20260808 06:08:39.780344 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.131:0
--local_ip_for_outbound_sockets=127.1.151.131
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:34155
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:43065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:08:39.890920 31016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:39.891146 31016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:39.891185 31016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:39.895138 31016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:39.895192 31016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:39.895249 31016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.131
I20260808 06:08:39.900269 31016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:43065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:34155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.131
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:39.901196 31016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:39.902141 31016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:08:39.911652 31023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:39.911619 31022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:39.911801 31016 server_base.cc:1034] running on GCE node
W20260808 06:08:39.911710 31025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:39.912241 31016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:39.912806 31016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:39.913960 31016 hybrid_clock.cc:648] HybridClock initialized: now 1786169319913934 us; error 39 us; skew 500 ppm
Aug 08 06:08:39 dist-test-slave-zj8q krb5kdc[30446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169319, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:08:39.916790 31016 init.cc:377] Logged in from keytab as kudu/127.1.151.131@KRBTEST.COM (short username kudu)
I20260808 06:08:39.917565 31016 webserver.cc:460] Webserver started at http://127.1.151.131:46285/ using document root <none> and password file <none>
I20260808 06:08:39.918097 31016 fs_manager.cc:359] Metadata directory not provided
I20260808 06:08:39.918192 31016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:08:39.918394 31016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:08:39.920356 31016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-2/data/instance:
uuid: "b2255b1dd7e64e87866c68dc76677d07"
format_stamp: "Formatted at 2026-08-08 06:08:39 on dist-test-slave-zj8q"
server_key: "78d4a638295c61d61d4eca5532c52d4c"
server_key_iv: "0281550101c1584a7239aaf7ead9c990"
server_key_version: "encryptionkey@0"
I20260808 06:08:39.920897 31016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance:
uuid: "b2255b1dd7e64e87866c68dc76677d07"
format_stamp: "Formatted at 2026-08-08 06:08:39 on dist-test-slave-zj8q"
server_key: "78d4a638295c61d61d4eca5532c52d4c"
server_key_iv: "0281550101c1584a7239aaf7ead9c990"
server_key_version: "encryptionkey@0"
I20260808 06:08:39.924446 31016 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 06:08:39.926832 31034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:39.927949 31016 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:08:39.928057 31016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-2/wal
uuid: "b2255b1dd7e64e87866c68dc76677d07"
format_stamp: "Formatted at 2026-08-08 06:08:39 on dist-test-slave-zj8q"
server_key: "78d4a638295c61d61d4eca5532c52d4c"
server_key_iv: "0281550101c1584a7239aaf7ead9c990"
server_key_version: "encryptionkey@0"
I20260808 06:08:39.928150 31016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:08:39.942345 31016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:08:39.945379 31016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:08:39.945569 31016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:08:39.946137 31016 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:08:39.946957 31016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:08:39.947003 31016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:39.947047 31016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:08:39.947074 31016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:39.955302 31016 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.131:35713
I20260808 06:08:39.955335 31204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.131:35713 every 8 connection(s)
I20260808 06:08:39.956161 31016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
Aug 08 06:08:39 dist-test-slave-zj8q krb5kdc[30446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169319, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:08:39.966400 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 31016
I20260808 06:08:39.966550 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance
I20260808 06:08:39.966907 1630 external_mini_cluster.cc:1428] Setting key 52fe8c1203764bfc3764e07f18ef0766
I20260808 06:08:39.969508 30571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:39.957890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:52811 (local address 127.1.151.190:34155)
0808 06:08:39.958033 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:39.958037 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:39.958854 (+ 817us) server_negotiation.cc:407] Connection header received
0808 06:08:39.960137 (+ 1283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:39.960141 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:39.960202 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:39.960279 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:39.961458 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:39.961997 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:39.963186 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:39.963403 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:39.963493 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:39.965810 (+ 2317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:39.965813 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:39.965815 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:08:39.965841 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:39.967333 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:39.967848 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:39.967852 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:39.967853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:39.967951 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:39.968322 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:39.968326 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:39.968328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:39.968534 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:39.968626 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:39.969260 (+ 634us) server_negotiation.cc:299] Negotiation successful
0808 06:08:39.969372 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 06:08:39.970440 31207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:39.958265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:34155 (local address 127.1.151.131:52811)
0808 06:08:39.958706 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:39.958748 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:39.959917 (+ 1169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:39.960465 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:39.960471 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:39.960822 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:08:39.961302 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:39.961316 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:39.962117 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:39.962120 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:39.963056 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:39.963067 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:39.963546 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:39.963550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:39.963713 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:39.964250 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:08:39.964266 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:08:39.965647 (+ 1381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:08:39.967440 (+ 1793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:08:39.967446 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:08:39.967460 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:08:39.967760 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:08:39.968041 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:08:39.968043 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:08:39.968045 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:08:39.968228 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:08:39.968698 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:08:39.968705 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:08:39.969016 (+ 311us) client_negotiation.cc:769] Sending connection context
0808 06:08:39.969189 (+ 173us) client_negotiation.cc:240] Negotiation successful
0808 06:08:39.969494 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":107,"threads_started":1}
I20260808 06:08:39.971486 31205 heartbeater.cc:347] Connected to a master server at 127.1.151.190:34155
I20260808 06:08:39.971712 31205 heartbeater.cc:464] Registering TS with master...
I20260808 06:08:39.972190 31205 heartbeater.cc:511] Master 127.1.151.190:34155 requested a full tablet report, sending...
I20260808 06:08:39.973163 30517 ts_manager.cc:194] Registered new tserver with Master: b2255b1dd7e64e87866c68dc76677d07 (127.1.151.131:35713)
I20260808 06:08:39.973712 30517 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.131@KRBTEST.COM'} at 127.1.151.131:52811
I20260808 06:08:39.980713 1630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 06:08:39.989645 30571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:39.982741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41968 (local address 127.1.151.190:34155)
0808 06:08:39.982852 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:39.982855 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:39.983062 (+ 207us) server_negotiation.cc:407] Connection header received
0808 06:08:39.983128 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:39.983130 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:39.983170 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:39.983230 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:39.983872 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:39.984396 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:39.985146 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:39.985355 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:39.985433 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:39.986899 (+ 1466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:39.986902 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:39.986904 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:08:39.986926 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:39.988230 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:39.988646 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:39.988648 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:39.988649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:39.988717 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:39.988942 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:39.988944 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:39.988945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:39.989145 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:39.989259 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:39.989440 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 06:08:39.989515 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 06:08:39.992553 30517 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41968:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 06:08:39.996038 30517 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 06:08:40.010528 31221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:40.004726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57448 (local address 127.1.151.129:41567)
0808 06:08:40.005569 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:40.005584 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:40.005610 (+ 26us) server_negotiation.cc:407] Connection header received
0808 06:08:40.005702 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:40.005707 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:40.005872 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:40.005988 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:08:40.006363 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:40.007411 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:40.009015 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:40.009854 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:40.009949 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:40.010104 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:40.010281 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 06:08:40.010377 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":717,"thread_start_us":93,"threads_started":1}
I20260808 06:08:40.010517 31217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:40.004331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:41567 (local address 127.0.0.1:57448)
0808 06:08:40.004653 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:40.004860 (+ 207us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:40.004989 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:40.006015 (+ 1026us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:40.006021 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:40.006057 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:08:40.006247 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:40.006255 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:40.007540 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:40.007543 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:40.008886 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:40.008896 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:40.009957 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:40.009960 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:40.010075 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:40.010113 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 06:08:40.010251 (+ 138us) client_negotiation.cc:240] Negotiation successful
0808 06:08:40.010372 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":626,"threads_started":1}
I20260808 06:08:40.011171 31218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:40.005203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:35713 (local address 127.0.0.1:54376)
0808 06:08:40.006553 (+ 1350us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:40.006565 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:40.006667 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:40.007128 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:40.007131 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:40.007151 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:08:40.007249 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:40.007255 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:40.008614 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:40.008618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:40.009507 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:40.009516 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:40.010722 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:40.010736 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:40.010878 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:40.010901 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 06:08:40.010971 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 06:08:40.011038 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1292,"mutex_wait_us":24,"thread_start_us":88,"threads_started":1}
I20260808 06:08:40.011240 31226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:40.005386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54376 (local address 127.1.151.131:35713)
0808 06:08:40.005749 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:40.005758 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:40.006698 (+ 940us) server_negotiation.cc:407] Connection header received
0808 06:08:40.006823 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:40.006828 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:40.006998 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:40.007143 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:08:40.007334 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:40.008466 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:40.009619 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:40.010426 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:40.010528 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:40.010657 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:40.011016 (+ 359us) server_negotiation.cc:299] Negotiation successful
0808 06:08:40.011104 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":177,"threads_started":1}
I20260808 06:08:40.011803 31223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:40.004429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:39773 (local address 127.0.0.1:49830)
0808 06:08:40.006261 (+ 1832us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:40.006272 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:40.006367 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:40.006906 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:40.006909 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:40.006927 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:08:40.007024 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:40.007030 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:40.008570 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:40.008574 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:40.009700 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:40.009712 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:40.010727 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:40.010732 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:40.011515 (+ 783us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:40.011537 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 06:08:40.011608 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 06:08:40.011673 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1774,"mutex_wait_us":69,"thread_start_us":97,"threads_started":1}
I20260808 06:08:40.011834 31224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:40.005227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49830 (local address 127.1.151.130:39773)
0808 06:08:40.005710 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:40.005722 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:40.006356 (+ 634us) server_negotiation.cc:407] Connection header received
0808 06:08:40.006468 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:40.006472 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:40.006602 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:40.006748 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:08:40.007379 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:40.008447 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:40.009829 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:40.010629 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:40.010729 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:40.010857 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:40.011642 (+ 785us) server_negotiation.cc:299] Negotiation successful
0808 06:08:40.011721 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":118,"threads_started":1}
I20260808 06:08:40.014087 31112 tablet_service.cc:1467] Processing CreateTablet for tablet 0752e9c6dbc240ecb6dda9818344f474 (DEFAULT_TABLE table=test-table [id=b402d5822b214ca3896db9b3d4779058]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:08:40.014111 30702 tablet_service.cc:1467] Processing CreateTablet for tablet 0752e9c6dbc240ecb6dda9818344f474 (DEFAULT_TABLE table=test-table [id=b402d5822b214ca3896db9b3d4779058]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:08:40.015151 31112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0752e9c6dbc240ecb6dda9818344f474. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:08:40.015233 30702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0752e9c6dbc240ecb6dda9818344f474. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:08:40.020820 30909 tablet_service.cc:1467] Processing CreateTablet for tablet 0752e9c6dbc240ecb6dda9818344f474 (DEFAULT_TABLE table=test-table [id=b402d5822b214ca3896db9b3d4779058]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:08:40.021940 30909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0752e9c6dbc240ecb6dda9818344f474. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:08:40.021925 31227 tablet_bootstrap.cc:492] T 0752e9c6dbc240ecb6dda9818344f474 P b2255b1dd7e64e87866c68dc76677d07: Bootstrap starting.
I20260808 06:08:40.023510 31229 tablet_bootstrap.cc:492] T 0752e9c6dbc240ecb6dda9818344f474 P e1a3c69a09a740ee8d6a10c32404d922: Bootstrap starting.
I20260808 06:08:40.024251 31227 tablet_bootstrap.cc:654] T 0752e9c6dbc240ecb6dda9818344f474 P b2255b1dd7e64e87866c68dc76677d07: Neither blocks nor log segments found. Creating new log.
I20260808 06:08:40.025262 31227 log.cc:824] T 0752e9c6dbc240ecb6dda9818344f474 P b2255b1dd7e64e87866c68dc76677d07: Log is configured to *not* fsync() on all Append() calls
I20260808 06:08:40.025588 31229 tablet_bootstrap.cc:654] T 0752e9c6dbc240ecb6dda9818344f474 P e1a3c69a09a740ee8d6a10c32404d922: Neither blocks nor log segments found. Creating new log.
I20260808 06:08:40.026831 31229 log.cc:824] T 0752e9c6dbc240ecb6dda9818344f474 P e1a3c69a09a740ee8d6a10c32404d922: Log is configured to *not* fsync() on all Append() calls
I20260808 06:08:40.027034 31227 tablet_bootstrap.cc:492] T 0752e9c6dbc240ecb6dda9818344f474 P b2255b1dd7e64e87866c68dc76677d07: No bootstrap required, opened a new log
I20260808 06:08:40.027227 31227 ts_tablet_manager.cc:1397] T 0752e9c6dbc240ecb6dda9818344f474 P b2255b1dd7e64e87866c68dc76677d07: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 06:08:40.028721 31229 tablet_bootstrap.cc:492] T 0752e9c6dbc240ecb6dda9818344f474 P e1a3c69a09a740ee8d6a10c32404d922: No bootstrap required, opened a new log
I20260808 06:08:40.028911 31229 ts_tablet_manager.cc:1397] T 0752e9c6dbc240ecb6dda9818344f474 P e1a3c69a09a740ee8d6a10c32404d922: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260808 06:08:40.029842 31233 tablet_bootstrap.cc:492] T 0752e9c6dbc240ecb6dda9818344f474 P 9e583c670f1747f9ad2aafe0bbf0bc3c: Bootstrap starting.
I20260808 06:08:40.031086 31227 raft_consensus.cc:348] T 0752e9c6dbc240ecb6dda9818344f474 P b2255b1dd7e64e87866c68dc76677d07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1a3c69a09a740ee8d6a10c32404d922" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 41567 } } peers { permanent_uuid: "9e583c670f1747f9ad2aafe0bbf0bc3c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 39773 } } peers { permanent_uuid: "b2255b1dd7e64e87866c68dc76677d07" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35713 } }
I20260808 06:08:40.031509 31227 raft_consensus.cc:374] T 0752e9c6dbc240ecb6dda9818344f474 P b2255b1dd7e64e87866c68dc76677d07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:08:40.031615 31227 raft_consensus.cc:729] T 0752e9c6dbc240ecb6dda9818344f474 P b2255b1dd7e64e87866c68dc76677d07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2255b1dd7e64e87866c68dc76677d07, State: Initialized, Role: FOLLOWER
I20260808 06:08:40.031808 31233 tablet_bootstrap.cc:654] T 0752e9c6dbc240ecb6dda9818344f474 P 9e583c670f1747f9ad2aafe0bbf0bc3c: Neither blocks nor log segments found. Creating new log.
I20260808 06:08:40.032176 31227 consensus_queue.cc:260] T 0752e9c6dbc240ecb6dda9818344f474 P b2255b1dd7e64e87866c68dc76677d07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1a3c69a09a740ee8d6a10c32404d922" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 41567 } } peers { permanent_uuid: "9e583c670f1747f9ad2aafe0bbf0bc3c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 39773 } } peers { permanent_uuid: "b2255b1dd7e64e87866c68dc76677d07" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35713 } }
I20260808 06:08:40.032972 31205 heartbeater.cc:503] Master 127.1.151.190:34155 was elected leader, sending a full tablet report...
I20260808 06:08:40.033222 31227 ts_tablet_manager.cc:1428] T 0752e9c6dbc240ecb6dda9818344f474 P b2255b1dd7e64e87866c68dc76677d07: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 06:08:40.033497 31233 log.cc:824] T 0752e9c6dbc240ecb6dda9818344f474 P 9e583c670f1747f9ad2aafe0bbf0bc3c: Log is configured to *not* fsync() on all Append() calls
I20260808 06:08:40.032685 31229 raft_consensus.cc:348] T 0752e9c6dbc240ecb6dda9818344f474 P e1a3c69a09a740ee8d6a10c32404d922 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1a3c69a09a740ee8d6a10c32404d922" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 41567 } } peers { permanent_uuid: "9e583c670f1747f9ad2aafe0bbf0bc3c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 39773 } } peers { permanent_uuid: "b2255b1dd7e64e87866c68dc76677d07" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35713 } }
I20260808 06:08:40.033972 31229 raft_consensus.cc:374] T 0752e9c6dbc240ecb6dda9818344f474 P e1a3c69a09a740ee8d6a10c32404d922 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:08:40.034026 31229 raft_consensus.cc:729] T 0752e9c6dbc240ecb6dda9818344f474 P e1a3c69a09a740ee8d6a10c32404d922 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1a3c69a09a740ee8d6a10c32404d922, State: Initialized, Role: FOLLOWER
I20260808 06:08:40.034473 31229 consensus_queue.cc:260] T 0752e9c6dbc240ecb6dda9818344f474 P e1a3c69a09a740ee8d6a10c32404d922 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1a3c69a09a740ee8d6a10c32404d922" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 41567 } } peers { permanent_uuid: "9e583c670f1747f9ad2aafe0bbf0bc3c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 39773 } } peers { permanent_uuid: "b2255b1dd7e64e87866c68dc76677d07" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35713 } }
I20260808 06:08:40.035104 30804 heartbeater.cc:503] Master 127.1.151.190:34155 was elected leader, sending a full tablet report...
I20260808 06:08:40.035414 31229 ts_tablet_manager.cc:1428] T 0752e9c6dbc240ecb6dda9818344f474 P e1a3c69a09a740ee8d6a10c32404d922: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260808 06:08:40.035863 30805 tablet.cc:2369] T 0752e9c6dbc240ecb6dda9818344f474 P e1a3c69a09a740ee8d6a10c32404d922: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 06:08:40.036088 31233 tablet_bootstrap.cc:492] T 0752e9c6dbc240ecb6dda9818344f474 P 9e583c670f1747f9ad2aafe0bbf0bc3c: No bootstrap required, opened a new log
I20260808 06:08:40.036271 31233 ts_tablet_manager.cc:1397] T 0752e9c6dbc240ecb6dda9818344f474 P 9e583c670f1747f9ad2aafe0bbf0bc3c: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260808 06:08:40.039333 31233 raft_consensus.cc:348] T 0752e9c6dbc240ecb6dda9818344f474 P 9e583c670f1747f9ad2aafe0bbf0bc3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1a3c69a09a740ee8d6a10c32404d922" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 41567 } } peers { permanent_uuid: "9e583c670f1747f9ad2aafe0bbf0bc3c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 39773 } } peers { permanent_uuid: "b2255b1dd7e64e87866c68dc76677d07" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35713 } }
I20260808 06:08:40.039556 31233 raft_consensus.cc:374] T 0752e9c6dbc240ecb6dda9818344f474 P 9e583c670f1747f9ad2aafe0bbf0bc3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:08:40.039611 31233 raft_consensus.cc:729] T 0752e9c6dbc240ecb6dda9818344f474 P 9e583c670f1747f9ad2aafe0bbf0bc3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e583c670f1747f9ad2aafe0bbf0bc3c, State: Initialized, Role: FOLLOWER
I20260808 06:08:40.039953 31233 consensus_queue.cc:260] T 0752e9c6dbc240ecb6dda9818344f474 P 9e583c670f1747f9ad2aafe0bbf0bc3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1a3c69a09a740ee8d6a10c32404d922" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 41567 } } peers { permanent_uuid: "9e583c670f1747f9ad2aafe0bbf0bc3c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 39773 } } peers { permanent_uuid: "b2255b1dd7e64e87866c68dc76677d07" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35713 } }
I20260808 06:08:40.040524 31007 heartbeater.cc:503] Master 127.1.151.190:34155 was elected leader, sending a full tablet report...
I20260808 06:08:40.040709 31233 ts_tablet_manager.cc:1428] T 0752e9c6dbc240ecb6dda9818344f474 P 9e583c670f1747f9ad2aafe0bbf0bc3c: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
W20260808 06:08:40.207093 31206 tablet.cc:2369] T 0752e9c6dbc240ecb6dda9818344f474 P b2255b1dd7e64e87866c68dc76677d07: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 06:08:40.248512 31012 tablet.cc:2369] T 0752e9c6dbc240ecb6dda9818344f474 P 9e583c670f1747f9ad2aafe0bbf0bc3c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 06:08:40.304891 31235 raft_consensus.cc:482] T 0752e9c6dbc240ecb6dda9818344f474 P b2255b1dd7e64e87866c68dc76677d07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:08:40.305152 31235 raft_consensus.cc:504] T 0752e9c6dbc240ecb6dda9818344f474 P b2255b1dd7e64e87866c68dc76677d07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1a3c69a09a740ee8d6a10c32404d922" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 41567 } } peers { permanent_uuid: "9e583c670f1747f9ad2aafe0bbf0bc3c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 39773 } } peers { permanent_uuid: "b2255b1dd7e64e87866c68dc76677d07" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35713 } }
I20260808 06:08:40.306407 31235 leader_election.cc:288] T 0752e9c6dbc240ecb6dda9818344f474 P b2255b1dd7e64e87866c68dc76677d07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1a3c69a09a740ee8d6a10c32404d922 (127.1.151.129:41567), 9e583c670f1747f9ad2aafe0bbf0bc3c (127.1.151.130:39773)
I20260808 06:08:40.310483 31207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:40.306683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:41567 (local address 127.1.151.131:50161)
0808 06:08:40.306799 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:40.306814 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:40.306910 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:40.307176 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:40.307179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:40.307198 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:08:40.307309 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:40.307314 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:40.308105 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:40.308109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:40.309198 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:40.309209 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:40.310045 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:40.310049 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:40.310138 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:40.310199 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 06:08:40.310273 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 06:08:40.310343 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 06:08:40.310519 31221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:40.306764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:50161 (local address 127.1.151.129:41567)
0808 06:08:40.306938 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:40.306942 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:40.306957 (+ 15us) server_negotiation.cc:407] Connection header received
0808 06:08:40.307014 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:40.307017 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:40.307069 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:40.307145 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:08:40.307414 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:40.307993 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:40.309335 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:40.309899 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:40.310017 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:40.310077 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:40.310323 (+ 246us) server_negotiation.cc:299] Negotiation successful
0808 06:08:40.310401 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 06:08:40.310778 31239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:40.306693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:39773 (local address 127.1.151.131:50047)
0808 06:08:40.307313 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:40.307324 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:40.307414 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:40.307627 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:40.307629 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:40.307645 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:08:40.307765 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:40.307770 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:40.308516 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:40.308520 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:40.309554 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:40.309562 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:40.310367 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:40.310370 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:40.310466 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:40.310488 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 06:08:40.310583 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 06:08:40.310650 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"mutex_wait_us":286,"thread_start_us":117,"threads_started":1}
I20260808 06:08:40.311224 31224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:40.306813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:50047 (local address 127.1.151.130:39773)
0808 06:08:40.306963 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:40.306967 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:40.307405 (+ 438us) server_negotiation.cc:407] Connection header received
0808 06:08:40.307501 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:40.307504 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:40.307550 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:40.307613 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:08:40.307869 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:40.308401 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:40.309672 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:40.310257 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:40.310363 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:40.310406 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:40.311006 (+ 600us) server_negotiation.cc:299] Negotiation successful
0808 06:08:40.311111 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 06:08:40.311148 30731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0752e9c6dbc240ecb6dda9818344f474" candidate_uuid: "b2255b1dd7e64e87866c68dc76677d07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1a3c69a09a740ee8d6a10c32404d922" is_pre_election: true
I20260808 06:08:40.311475 30731 raft_consensus.cc:2454] T 0752e9c6dbc240ecb6dda9818344f474 P e1a3c69a09a740ee8d6a10c32404d922 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2255b1dd7e64e87866c68dc76677d07 in term 0.
I20260808 06:08:40.311816 30943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0752e9c6dbc240ecb6dda9818344f474" candidate_uuid: "b2255b1dd7e64e87866c68dc76677d07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e583c670f1747f9ad2aafe0bbf0bc3c" is_pre_election: true
I20260808 06:08:40.312023 31037 leader_election.cc:302] T 0752e9c6dbc240ecb6dda9818344f474 P b2255b1dd7e64e87866c68dc76677d07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2255b1dd7e64e87866c68dc76677d07, e1a3c69a09a740ee8d6a10c32404d922; no voters:
I20260808 06:08:40.312072 30943 raft_consensus.cc:2454] T 0752e9c6dbc240ecb6dda9818344f474 P 9e583c670f1747f9ad2aafe0bbf0bc3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2255b1dd7e64e87866c68dc76677d07 in term 0.
I20260808 06:08:40.312316 31235 raft_consensus.cc:2781] T 0752e9c6dbc240ecb6dda9818344f474 P b2255b1dd7e64e87866c68dc76677d07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 06:08:40.312424 31235 raft_consensus.cc:482] T 0752e9c6dbc240ecb6dda9818344f474 P b2255b1dd7e64e87866c68dc76677d07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 06:08:40.312480 31235 raft_consensus.cc:3037] T 0752e9c6dbc240ecb6dda9818344f474 P b2255b1dd7e64e87866c68dc76677d07 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:08:40.313484 31235 raft_consensus.cc:504] T 0752e9c6dbc240ecb6dda9818344f474 P b2255b1dd7e64e87866c68dc76677d07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1a3c69a09a740ee8d6a10c32404d922" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 41567 } } peers { permanent_uuid: "9e583c670f1747f9ad2aafe0bbf0bc3c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 39773 } } peers { permanent_uuid: "b2255b1dd7e64e87866c68dc76677d07" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35713 } }
I20260808 06:08:40.313815 31235 leader_election.cc:288] T 0752e9c6dbc240ecb6dda9818344f474 P b2255b1dd7e64e87866c68dc76677d07 [CANDIDATE]: Term 1 election: Requested vote from peers e1a3c69a09a740ee8d6a10c32404d922 (127.1.151.129:41567), 9e583c670f1747f9ad2aafe0bbf0bc3c (127.1.151.130:39773)
I20260808 06:08:40.314173 30943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0752e9c6dbc240ecb6dda9818344f474" candidate_uuid: "b2255b1dd7e64e87866c68dc76677d07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e583c670f1747f9ad2aafe0bbf0bc3c"
I20260808 06:08:40.314199 30731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0752e9c6dbc240ecb6dda9818344f474" candidate_uuid: "b2255b1dd7e64e87866c68dc76677d07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1a3c69a09a740ee8d6a10c32404d922"
I20260808 06:08:40.314291 30943 raft_consensus.cc:3037] T 0752e9c6dbc240ecb6dda9818344f474 P 9e583c670f1747f9ad2aafe0bbf0bc3c [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:08:40.314327 30731 raft_consensus.cc:3037] T 0752e9c6dbc240ecb6dda9818344f474 P e1a3c69a09a740ee8d6a10c32404d922 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:08:40.315332 30943 raft_consensus.cc:2454] T 0752e9c6dbc240ecb6dda9818344f474 P 9e583c670f1747f9ad2aafe0bbf0bc3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2255b1dd7e64e87866c68dc76677d07 in term 1.
I20260808 06:08:40.315446 30731 raft_consensus.cc:2454] T 0752e9c6dbc240ecb6dda9818344f474 P e1a3c69a09a740ee8d6a10c32404d922 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2255b1dd7e64e87866c68dc76677d07 in term 1.
I20260808 06:08:40.315639 31038 leader_election.cc:302] T 0752e9c6dbc240ecb6dda9818344f474 P b2255b1dd7e64e87866c68dc76677d07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e583c670f1747f9ad2aafe0bbf0bc3c, b2255b1dd7e64e87866c68dc76677d07; no voters:
I20260808 06:08:40.315805 31235 raft_consensus.cc:2781] T 0752e9c6dbc240ecb6dda9818344f474 P b2255b1dd7e64e87866c68dc76677d07 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:08:40.315999 31235 raft_consensus.cc:686] T 0752e9c6dbc240ecb6dda9818344f474 P b2255b1dd7e64e87866c68dc76677d07 [term 1 LEADER]: Becoming Leader. State: Replica: b2255b1dd7e64e87866c68dc76677d07, State: Running, Role: LEADER
I20260808 06:08:40.316335 31235 consensus_queue.cc:237] T 0752e9c6dbc240ecb6dda9818344f474 P b2255b1dd7e64e87866c68dc76677d07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1a3c69a09a740ee8d6a10c32404d922" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 41567 } } peers { permanent_uuid: "9e583c670f1747f9ad2aafe0bbf0bc3c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 39773 } } peers { permanent_uuid: "b2255b1dd7e64e87866c68dc76677d07" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35713 } }
I20260808 06:08:40.319401 30517 catalog_manager.cc:5495] T 0752e9c6dbc240ecb6dda9818344f474 P b2255b1dd7e64e87866c68dc76677d07 reported cstate change: term changed from 0 to 1, leader changed from <none> to b2255b1dd7e64e87866c68dc76677d07 (127.1.151.131). New cstate: current_term: 1 leader_uuid: "b2255b1dd7e64e87866c68dc76677d07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1a3c69a09a740ee8d6a10c32404d922" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 41567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e583c670f1747f9ad2aafe0bbf0bc3c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 39773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2255b1dd7e64e87866c68dc76677d07" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 35713 } health_report { overall_health: HEALTHY } } }
I20260808 06:08:40.378468 31226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:40.375318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54384 (local address 127.1.151.131:35713)
0808 06:08:40.375437 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:40.375441 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:40.375454 (+ 13us) server_negotiation.cc:407] Connection header received
0808 06:08:40.375548 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:40.375551 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:40.375605 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:40.375698 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:08:40.375907 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:40.376399 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:40.377190 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:40.377393 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:40.377461 (+ 68us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:40.377725 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:08:40.378115 (+ 390us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:08:40.378256 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:40.378314 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 06:08:40.378352 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 06:08:40.384400 30731 raft_consensus.cc:1261] T 0752e9c6dbc240ecb6dda9818344f474 P e1a3c69a09a740ee8d6a10c32404d922 [term 1 FOLLOWER]: Refusing update from remote peer b2255b1dd7e64e87866c68dc76677d07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:08:40.384400 30943 raft_consensus.cc:1261] T 0752e9c6dbc240ecb6dda9818344f474 P 9e583c670f1747f9ad2aafe0bbf0bc3c [term 1 FOLLOWER]: Refusing update from remote peer b2255b1dd7e64e87866c68dc76677d07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:08:40.385107 31240 consensus_queue.cc:1035] T 0752e9c6dbc240ecb6dda9818344f474 P b2255b1dd7e64e87866c68dc76677d07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1a3c69a09a740ee8d6a10c32404d922" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 41567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:08:40.385310 31235 consensus_queue.cc:1035] T 0752e9c6dbc240ecb6dda9818344f474 P b2255b1dd7e64e87866c68dc76677d07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e583c670f1747f9ad2aafe0bbf0bc3c" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 39773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:08:40.394603 31245 mvcc.cc:205] Tried to move back new op lower bound from 7316149536287453184 to 7316149536018173952. Current Snapshot: MvccSnapshot[applied={T|T < 7316149536287453184 or (T in {7316149536287453184})}]
I20260808 06:08:40.400477 30517 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:08:40.400628 30517 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:08:40.402812 30517 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8eaa3d2db5c646f1b25e541afd445f34: Sending DeleteTablet for 3 replicas of tablet 0752e9c6dbc240ecb6dda9818344f474
I20260808 06:08:40.403591 30909 tablet_service.cc:1514] Processing DeleteTablet for tablet 0752e9c6dbc240ecb6dda9818344f474 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:08:40 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:49830
I20260808 06:08:40.403785 31112 tablet_service.cc:1514] Processing DeleteTablet for tablet 0752e9c6dbc240ecb6dda9818344f474 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:08:40 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:54376
I20260808 06:08:40.404124 30702 tablet_service.cc:1514] Processing DeleteTablet for tablet 0752e9c6dbc240ecb6dda9818344f474 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:08:40 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:57448
I20260808 06:08:40.404204 31260 tablet_replica.cc:330] T 0752e9c6dbc240ecb6dda9818344f474 P b2255b1dd7e64e87866c68dc76677d07: stopping tablet replica
I20260808 06:08:40.404358 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 30608
I20260808 06:08:40.404570 31260 raft_consensus.cc:2229] T 0752e9c6dbc240ecb6dda9818344f474 P b2255b1dd7e64e87866c68dc76677d07 [term 1 LEADER]: Raft consensus shutting down.
I20260808 06:08:40.405018 31260 raft_consensus.cc:2258] T 0752e9c6dbc240ecb6dda9818344f474 P b2255b1dd7e64e87866c68dc76677d07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:08:40.411615 31260 ts_tablet_manager.cc:1905] T 0752e9c6dbc240ecb6dda9818344f474 P b2255b1dd7e64e87866c68dc76677d07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:08:40.414556 31259 tablet_replica.cc:330] T 0752e9c6dbc240ecb6dda9818344f474 P 9e583c670f1747f9ad2aafe0bbf0bc3c: stopping tablet replica
I20260808 06:08:40.414599 31260 ts_tablet_manager.cc:1918] T 0752e9c6dbc240ecb6dda9818344f474 P b2255b1dd7e64e87866c68dc76677d07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:08:40.414767 31260 log.cc:1196] T 0752e9c6dbc240ecb6dda9818344f474 P b2255b1dd7e64e87866c68dc76677d07: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-2/wal/wals/0752e9c6dbc240ecb6dda9818344f474
I20260808 06:08:40.415004 31259 raft_consensus.cc:2229] T 0752e9c6dbc240ecb6dda9818344f474 P 9e583c670f1747f9ad2aafe0bbf0bc3c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:08:40.415165 31260 ts_tablet_manager.cc:1939] T 0752e9c6dbc240ecb6dda9818344f474 P b2255b1dd7e64e87866c68dc76677d07: Deleting consensus metadata
I20260808 06:08:40.415539 31259 raft_consensus.cc:2258] T 0752e9c6dbc240ecb6dda9818344f474 P 9e583c670f1747f9ad2aafe0bbf0bc3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:08:40.416494 30499 catalog_manager.cc:4841] TS b2255b1dd7e64e87866c68dc76677d07 (127.1.151.131:35713): tablet 0752e9c6dbc240ecb6dda9818344f474 (table test-table [id=b402d5822b214ca3896db9b3d4779058]) successfully deleted
I20260808 06:08:40.417909 31259 ts_tablet_manager.cc:1905] T 0752e9c6dbc240ecb6dda9818344f474 P 9e583c670f1747f9ad2aafe0bbf0bc3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:08:40.420742 31259 ts_tablet_manager.cc:1918] T 0752e9c6dbc240ecb6dda9818344f474 P 9e583c670f1747f9ad2aafe0bbf0bc3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:08:40.420842 31259 log.cc:1196] T 0752e9c6dbc240ecb6dda9818344f474 P 9e583c670f1747f9ad2aafe0bbf0bc3c: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786169070948234-1630-0/minicluster-data/ts-1/wal/wals/0752e9c6dbc240ecb6dda9818344f474
I20260808 06:08:40.421125 31259 ts_tablet_manager.cc:1939] T 0752e9c6dbc240ecb6dda9818344f474 P 9e583c670f1747f9ad2aafe0bbf0bc3c: Deleting consensus metadata
I20260808 06:08:40.422376 30500 catalog_manager.cc:4841] TS 9e583c670f1747f9ad2aafe0bbf0bc3c (127.1.151.130:39773): tablet 0752e9c6dbc240ecb6dda9818344f474 (table test-table [id=b402d5822b214ca3896db9b3d4779058]) successfully deleted
W20260808 06:08:40.425799 30499 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.151.129:41567 (error 108)
I20260808 06:08:40.426050 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 30810
I20260808 06:08:40.432163 31223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:40.431838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:41567 (local address 127.0.0.1:57460)
0808 06:08:40.432005 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:40.432064 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.129:41567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
W20260808 06:08:40.432320 30499 catalog_manager.cc:4584] TS e1a3c69a09a740ee8d6a10c32404d922 (127.1.151.129:41567): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0752e9c6dbc240ecb6dda9818344f474: Network error: Client connection negotiation failed: client connection to 127.1.151.129:41567: connect: Connection refused (error 111)
I20260808 06:08:40.443363 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 31016
I20260808 06:08:40.453816 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 30466
2026-08-08T06:08:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2649 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T06:08:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:08:40Z Disabled control of system clock
I20260808 06:08:40.492614 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:39173
--webserver_interface=127.1.151.190
--webserver_port=0
--builtin_ntp_servers=127.1.151.148:38585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:39173
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 06:08:40.601016 31270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:40.601277 31270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:40.601311 31270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:40.605214 31270 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:08:40.605264 31270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:40.605278 31270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:40.605295 31270 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:08:40.605324 31270 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:08:40.610118 31270 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:38585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:39173
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:39173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:40.611136 31270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:40.612438 31270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:08:40.620671 31286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:40.620688 31270 server_base.cc:1034] running on GCE node
W20260808 06:08:40.620702 31284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:40.620718 31281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:40.621449 31270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:40.622506 31270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:40.623663 31270 hybrid_clock.cc:648] HybridClock initialized: now 1786169320623650 us; error 24 us; skew 500 ppm
I20260808 06:08:40.625384 31270 webserver.cc:460] Webserver started at http://127.1.151.190:34319/ using document root <none> and password file <none>
I20260808 06:08:40.625921 31270 fs_manager.cc:359] Metadata directory not provided
I20260808 06:08:40.625988 31270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:08:40.626209 31270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:08:40.628124 31270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "6d7835e3098f4fda8ca87ab710985cf6"
format_stamp: "Formatted at 2026-08-08 06:08:40 on dist-test-slave-zj8q"
server_key: "b954047f68030751e9ee1d050262845c"
server_key_iv: "8a12341dcc1f08847d0b225208f0bd1f"
server_key_version: "encryptionkey@0"
I20260808 06:08:40.628664 31270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "6d7835e3098f4fda8ca87ab710985cf6"
format_stamp: "Formatted at 2026-08-08 06:08:40 on dist-test-slave-zj8q"
server_key: "b954047f68030751e9ee1d050262845c"
server_key_iv: "8a12341dcc1f08847d0b225208f0bd1f"
server_key_version: "encryptionkey@0"
I20260808 06:08:40.632090 31270 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260808 06:08:40.634413 31294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:40.635560 31270 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 06:08:40.635677 31270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "6d7835e3098f4fda8ca87ab710985cf6"
format_stamp: "Formatted at 2026-08-08 06:08:40 on dist-test-slave-zj8q"
server_key: "b954047f68030751e9ee1d050262845c"
server_key_iv: "8a12341dcc1f08847d0b225208f0bd1f"
server_key_version: "encryptionkey@0"
I20260808 06:08:40.635778 31270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:08:40.660058 31270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:08:40.660703 31270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:08:40.660871 31270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:08:40.667414 31270 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:39173
I20260808 06:08:40.667429 31375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:39173 every 8 connection(s)
I20260808 06:08:40.668303 31270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:08:40.672104 31377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:08:40.677889 31377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d7835e3098f4fda8ca87ab710985cf6: Bootstrap starting.
I20260808 06:08:40.678382 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 31270
I20260808 06:08:40.678503 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:08:40.678752 1630 external_mini_cluster.cc:1428] Setting key 937e2e5542292d7bc3c4372f2848ae76
I20260808 06:08:40.680524 31377 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d7835e3098f4fda8ca87ab710985cf6: Neither blocks nor log segments found. Creating new log.
I20260808 06:08:40.681246 31377 log.cc:824] T 00000000000000000000000000000000 P 6d7835e3098f4fda8ca87ab710985cf6: Log is configured to *not* fsync() on all Append() calls
I20260808 06:08:40.683357 31377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d7835e3098f4fda8ca87ab710985cf6: No bootstrap required, opened a new log
I20260808 06:08:40.687424 31377 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d7835e3098f4fda8ca87ab710985cf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7835e3098f4fda8ca87ab710985cf6" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39173 } }
I20260808 06:08:40.687656 31377 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d7835e3098f4fda8ca87ab710985cf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:08:40.687705 31377 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d7835e3098f4fda8ca87ab710985cf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d7835e3098f4fda8ca87ab710985cf6, State: Initialized, Role: FOLLOWER
I20260808 06:08:40.688012 31385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:40.680016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59416 (local address 127.1.151.190:39173)
0808 06:08:40.680670 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:40.680686 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:40.680721 (+ 35us) server_negotiation.cc:407] Connection header received
0808 06:08:40.681341 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:40.681366 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:40.681840 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:40.682195 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:40.682548 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:40.683595 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:40.684795 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:40.685265 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:40.685393 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:40.686253 (+ 860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:40.686258 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:40.686270 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:08:40.686666 (+ 396us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:40.686720 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:08:40.686740 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:40.686815 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:40.686953 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 06:08:40.687205 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":126,"threads_started":1}
I20260808 06:08:40.688112 31377 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d7835e3098f4fda8ca87ab710985cf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7835e3098f4fda8ca87ab710985cf6" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39173 } }
I20260808 06:08:40.688258 31377 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d7835e3098f4fda8ca87ab710985cf6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:08:40.688321 31377 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d7835e3098f4fda8ca87ab710985cf6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:08:40.688414 31377 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d7835e3098f4fda8ca87ab710985cf6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:08:40.689273 31377 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d7835e3098f4fda8ca87ab710985cf6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7835e3098f4fda8ca87ab710985cf6" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39173 } }
I20260808 06:08:40.689608 31377 leader_election.cc:302] T 00000000000000000000000000000000 P 6d7835e3098f4fda8ca87ab710985cf6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d7835e3098f4fda8ca87ab710985cf6; no voters:
I20260808 06:08:40.689944 31377 leader_election.cc:288] T 00000000000000000000000000000000 P 6d7835e3098f4fda8ca87ab710985cf6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:08:40.690057 31387 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d7835e3098f4fda8ca87ab710985cf6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:08:40.690403 31387 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d7835e3098f4fda8ca87ab710985cf6 [term 1 LEADER]: Becoming Leader. State: Replica: 6d7835e3098f4fda8ca87ab710985cf6, State: Running, Role: LEADER
I20260808 06:08:40.690759 31387 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d7835e3098f4fda8ca87ab710985cf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7835e3098f4fda8ca87ab710985cf6" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39173 } }
I20260808 06:08:40.691288 31377 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d7835e3098f4fda8ca87ab710985cf6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:08:40.692456 31388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d7835e3098f4fda8ca87ab710985cf6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d7835e3098f4fda8ca87ab710985cf6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7835e3098f4fda8ca87ab710985cf6" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39173 } } }
I20260808 06:08:40.692584 31388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d7835e3098f4fda8ca87ab710985cf6 [sys.catalog]: This master's current role is: LEADER
I20260808 06:08:40.692902 31389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d7835e3098f4fda8ca87ab710985cf6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d7835e3098f4fda8ca87ab710985cf6. Latest consensus state: current_term: 1 leader_uuid: "6d7835e3098f4fda8ca87ab710985cf6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7835e3098f4fda8ca87ab710985cf6" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 39173 } } }
I20260808 06:08:40.692992 31389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d7835e3098f4fda8ca87ab710985cf6 [sys.catalog]: This master's current role is: LEADER
I20260808 06:08:40.693132 31398 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:08:40.695816 31398 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:08:40.700958 31398 catalog_manager.cc:1347] Generated new cluster ID: ca6007e9334b49e9b2325a9ad6c0b839
I20260808 06:08:40.701026 31398 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:08:40.711589 31398 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:08:40.712741 31398 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:08:40.721760 31398 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d7835e3098f4fda8ca87ab710985cf6: Generated new TSK 0
I20260808 06:08:40.722517 31398 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 06:08:40.730463 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.129:0
--local_ip_for_outbound_sockets=127.1.151.129
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:39173
--builtin_ntp_servers=127.1.151.148:38585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 06:08:40.841648 31419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:40.841883 31419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:40.841919 31419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:40.845834 31419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:40.845881 31419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:40.845937 31419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.129
I20260808 06:08:40.850584 31419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:38585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:39173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.129
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:40.851570 31419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:40.852913 31419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:08:40.862169 31427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:40.862303 31419 server_base.cc:1034] running on GCE node
W20260808 06:08:40.862246 31428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:40.862187 31430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:40.862793 31419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:40.863389 31419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:40.864540 31419 hybrid_clock.cc:648] HybridClock initialized: now 1786169320864541 us; error 54 us; skew 500 ppm
I20260808 06:08:40.866442 31419 webserver.cc:460] Webserver started at http://127.1.151.129:34409/ using document root <none> and password file <none>
I20260808 06:08:40.867022 31419 fs_manager.cc:359] Metadata directory not provided
I20260808 06:08:40.867089 31419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:08:40.867302 31419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:08:40.869324 31419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-0/data/instance:
uuid: "8a250dadf3e74a4dabe8705d2547ea3b"
format_stamp: "Formatted at 2026-08-08 06:08:40 on dist-test-slave-zj8q"
server_key: "6eda6b3b6242cac0f654fe98d0e011c8"
server_key_iv: "8a85b40679ceb344433a59a480cf4438"
server_key_version: "encryptionkey@0"
I20260808 06:08:40.869866 31419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance:
uuid: "8a250dadf3e74a4dabe8705d2547ea3b"
format_stamp: "Formatted at 2026-08-08 06:08:40 on dist-test-slave-zj8q"
server_key: "6eda6b3b6242cac0f654fe98d0e011c8"
server_key_iv: "8a85b40679ceb344433a59a480cf4438"
server_key_version: "encryptionkey@0"
I20260808 06:08:40.873430 31419 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 06:08:40.875876 31440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:40.877015 31419 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:08:40.877147 31419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-0/wal
uuid: "8a250dadf3e74a4dabe8705d2547ea3b"
format_stamp: "Formatted at 2026-08-08 06:08:40 on dist-test-slave-zj8q"
server_key: "6eda6b3b6242cac0f654fe98d0e011c8"
server_key_iv: "8a85b40679ceb344433a59a480cf4438"
server_key_version: "encryptionkey@0"
I20260808 06:08:40.877240 31419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:08:40.894421 31419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:08:40.895042 31419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:08:40.895211 31419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:08:40.895768 31419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:08:40.896672 31419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:08:40.896721 31419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:40.896775 31419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:08:40.896803 31419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:40.905400 31419 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.129:42213
I20260808 06:08:40.905450 31614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.129:42213 every 8 connection(s)
I20260808 06:08:40.906349 31419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
I20260808 06:08:40.907509 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 31419
I20260808 06:08:40.907645 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance
I20260808 06:08:40.907958 1630 external_mini_cluster.cc:1428] Setting key 44f041114868e0eadc7ed4b2faca3be2
I20260808 06:08:40.910130 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.130:0
--local_ip_for_outbound_sockets=127.1.151.130
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:39173
--builtin_ntp_servers=127.1.151.148:38585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 06:08:40.916239 31385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:40.908540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:35971 (local address 127.1.151.190:39173)
0808 06:08:40.908704 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:40.908709 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:40.909701 (+ 992us) server_negotiation.cc:407] Connection header received
0808 06:08:40.911451 (+ 1750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:40.911454 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:40.911500 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:40.911579 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:40.912564 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:40.913096 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:40.914260 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:40.914609 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:40.914715 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:40.915755 (+ 1040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:40.915760 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:40.915763 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:08:40.915789 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:40.915816 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:08:40.915828 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:40.915896 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:40.916019 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 06:08:40.916085 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 06:08:40.917021 31617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:40.908772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39173 (local address 127.1.151.129:35971)
0808 06:08:40.909524 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:40.909579 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:40.911218 (+ 1639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:40.911758 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:40.911766 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:40.911873 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:08:40.912423 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:40.912437 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:40.913230 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:40.913234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:40.914115 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:40.914125 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:40.914756 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:40.914759 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:40.914917 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:40.915489 (+ 572us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 06:08:40.915490 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:08:40.915528 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 06:08:40.915589 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:08:40.915906 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:08:40.915911 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:08:40.915914 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 06:08:40.916002 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 06:08:40.916246 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":151,"threads_started":1}
I20260808 06:08:40.918198 31615 heartbeater.cc:347] Connected to a master server at 127.1.151.190:39173
I20260808 06:08:40.918416 31615 heartbeater.cc:464] Registering TS with master...
I20260808 06:08:40.918906 31615 heartbeater.cc:511] Master 127.1.151.190:39173 requested a full tablet report, sending...
I20260808 06:08:40.920358 31315 ts_manager.cc:194] Registered new tserver with Master: 8a250dadf3e74a4dabe8705d2547ea3b (127.1.151.129:42213)
I20260808 06:08:40.921489 31315 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.151.129:35971
W20260808 06:08:41.023931 31618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:41.024195 31618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:41.024232 31618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:41.028317 31618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:41.028370 31618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:41.028430 31618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.130
I20260808 06:08:41.033152 31618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:38585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:39173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.130
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:41.034212 31618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:41.035554 31618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:08:41.044930 31628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:41.044930 31627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:41.044951 31632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:41.045078 31618 server_base.cc:1034] running on GCE node
I20260808 06:08:41.045562 31618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:41.046201 31618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:41.047359 31618 hybrid_clock.cc:648] HybridClock initialized: now 1786169321047336 us; error 41 us; skew 500 ppm
I20260808 06:08:41.049111 31618 webserver.cc:460] Webserver started at http://127.1.151.130:36697/ using document root <none> and password file <none>
I20260808 06:08:41.049688 31618 fs_manager.cc:359] Metadata directory not provided
I20260808 06:08:41.049758 31618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:08:41.049940 31618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:08:41.052014 31618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-1/data/instance:
uuid: "be8b6618164f4c6584db75195ccbf714"
format_stamp: "Formatted at 2026-08-08 06:08:41 on dist-test-slave-zj8q"
server_key: "26f4cad0fe1ccd606ee4999f47a7a8df"
server_key_iv: "f84bb34d8561c575e8fc018fb70ffda6"
server_key_version: "encryptionkey@0"
I20260808 06:08:41.052562 31618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance:
uuid: "be8b6618164f4c6584db75195ccbf714"
format_stamp: "Formatted at 2026-08-08 06:08:41 on dist-test-slave-zj8q"
server_key: "26f4cad0fe1ccd606ee4999f47a7a8df"
server_key_iv: "f84bb34d8561c575e8fc018fb70ffda6"
server_key_version: "encryptionkey@0"
I20260808 06:08:41.056180 31618 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 06:08:41.058625 31650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:41.059792 31618 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:08:41.059908 31618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-1/wal
uuid: "be8b6618164f4c6584db75195ccbf714"
format_stamp: "Formatted at 2026-08-08 06:08:41 on dist-test-slave-zj8q"
server_key: "26f4cad0fe1ccd606ee4999f47a7a8df"
server_key_iv: "f84bb34d8561c575e8fc018fb70ffda6"
server_key_version: "encryptionkey@0"
I20260808 06:08:41.060016 31618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:08:41.091635 31618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:08:41.092434 31618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:08:41.092614 31618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:08:41.093235 31618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:08:41.094240 31618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:08:41.094290 31618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:41.094329 31618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:08:41.094349 31618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:41.102784 31618 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.130:37479
I20260808 06:08:41.102874 31822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.130:37479 every 8 connection(s)
I20260808 06:08:41.103885 31618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
I20260808 06:08:41.108768 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 31618
I20260808 06:08:41.108875 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance
I20260808 06:08:41.109143 1630 external_mini_cluster.cc:1428] Setting key 0cdee0fad436e74a44ceb3b56d8d82f5
I20260808 06:08:41.111102 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.131:0
--local_ip_for_outbound_sockets=127.1.151.131
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:39173
--builtin_ntp_servers=127.1.151.148:38585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 06:08:41.113183 31385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:41.105640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:40011 (local address 127.1.151.190:39173)
0808 06:08:41.105765 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:41.105769 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:41.106862 (+ 1093us) server_negotiation.cc:407] Connection header received
0808 06:08:41.108271 (+ 1409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:41.108275 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:41.108334 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:41.108406 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:41.109382 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:41.110030 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:41.111352 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:41.111582 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:41.111671 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:41.112731 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:41.112734 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:41.112737 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:08:41.112751 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:41.112772 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:08:41.112780 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:41.112837 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:41.112977 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 06:08:41.113039 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 06:08:41.114035 31827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:41.106015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39173 (local address 127.1.151.130:40011)
0808 06:08:41.106697 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:41.106757 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:41.108029 (+ 1272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:41.108579 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:41.108586 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:41.108669 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:08:41.109197 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:41.109215 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:41.110175 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:41.110179 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:41.111193 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:41.111206 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:41.111728 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:41.111732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:41.111906 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:41.112487 (+ 581us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 06:08:41.112488 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:08:41.112527 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 06:08:41.112589 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:08:41.112849 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:08:41.112859 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:08:41.112865 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 06:08:41.112968 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 06:08:41.113234 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":161,"threads_started":1}
I20260808 06:08:41.115134 31823 heartbeater.cc:347] Connected to a master server at 127.1.151.190:39173
I20260808 06:08:41.115340 31823 heartbeater.cc:464] Registering TS with master...
I20260808 06:08:41.115833 31823 heartbeater.cc:511] Master 127.1.151.190:39173 requested a full tablet report, sending...
I20260808 06:08:41.116887 31315 ts_manager.cc:194] Registered new tserver with Master: be8b6618164f4c6584db75195ccbf714 (127.1.151.130:37479)
I20260808 06:08:41.117628 31315 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.151.130:40011
W20260808 06:08:41.222375 31828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:41.222611 31828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:41.222647 31828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:41.226498 31828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:41.226538 31828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:41.226588 31828 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.131
I20260808 06:08:41.231117 31828 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:38585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:39173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.131
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:41.232038 31828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:41.233289 31828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:08:41.242489 31837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:41.242501 31835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:41.242630 31843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:41.242944 31828 server_base.cc:1034] running on GCE node
I20260808 06:08:41.243319 31828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:41.243921 31828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:41.245077 31828 hybrid_clock.cc:648] HybridClock initialized: now 1786169321245053 us; error 33 us; skew 500 ppm
I20260808 06:08:41.246989 31828 webserver.cc:460] Webserver started at http://127.1.151.131:45421/ using document root <none> and password file <none>
I20260808 06:08:41.247570 31828 fs_manager.cc:359] Metadata directory not provided
I20260808 06:08:41.247641 31828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:08:41.247848 31828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:08:41.249866 31828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-2/data/instance:
uuid: "0984588df55342b99f705767aba1c182"
format_stamp: "Formatted at 2026-08-08 06:08:41 on dist-test-slave-zj8q"
server_key: "7b75f4232aac4c267716c330e066635d"
server_key_iv: "7a20768d36a0dcc7ff9adee67fb2f820"
server_key_version: "encryptionkey@0"
I20260808 06:08:41.250448 31828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance:
uuid: "0984588df55342b99f705767aba1c182"
format_stamp: "Formatted at 2026-08-08 06:08:41 on dist-test-slave-zj8q"
server_key: "7b75f4232aac4c267716c330e066635d"
server_key_iv: "7a20768d36a0dcc7ff9adee67fb2f820"
server_key_version: "encryptionkey@0"
I20260808 06:08:41.254012 31828 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 06:08:41.256400 31853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:41.257488 31828 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:08:41.257607 31828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-2/wal
uuid: "0984588df55342b99f705767aba1c182"
format_stamp: "Formatted at 2026-08-08 06:08:41 on dist-test-slave-zj8q"
server_key: "7b75f4232aac4c267716c330e066635d"
server_key_iv: "7a20768d36a0dcc7ff9adee67fb2f820"
server_key_version: "encryptionkey@0"
I20260808 06:08:41.257709 31828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:08:41.276106 31828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:08:41.276775 31828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:08:41.276949 31828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:08:41.277510 31828 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:08:41.278458 31828 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:08:41.278506 31828 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:41.278570 31828 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:08:41.278595 31828 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:41.286980 31828 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.131:40719
I20260808 06:08:41.287014 32029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.131:40719 every 8 connection(s)
I20260808 06:08:41.287860 31828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
I20260808 06:08:41.288405 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 31828
I20260808 06:08:41.288513 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance
I20260808 06:08:41.288810 1630 external_mini_cluster.cc:1428] Setting key 515fde090086660c5d3ce91aca4c4977
I20260808 06:08:41.297106 31385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:41.289845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:33593 (local address 127.1.151.190:39173)
0808 06:08:41.289974 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:41.289978 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:41.291180 (+ 1202us) server_negotiation.cc:407] Connection header received
0808 06:08:41.292613 (+ 1433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:41.292617 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:41.292680 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:41.292773 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:41.293622 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:41.294174 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:41.295292 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:41.295524 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:41.295611 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:41.296661 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:41.296665 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:41.296668 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:08:41.296681 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:41.296700 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:08:41.296708 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:41.296765 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:41.296905 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 06:08:41.296966 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 06:08:41.297922 32034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:41.290316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:39173 (local address 127.1.151.131:33593)
0808 06:08:41.291036 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:41.291075 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:41.292414 (+ 1339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:41.292926 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:41.292933 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:41.293013 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:08:41.293486 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:41.293498 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:41.294302 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:41.294305 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:41.295174 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:41.295184 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:41.295662 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:41.295667 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:41.295828 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:41.296412 (+ 584us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 06:08:41.296413 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:08:41.296452 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 06:08:41.296516 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:08:41.296777 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:08:41.296787 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:08:41.296793 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 06:08:41.296895 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 06:08:41.297175 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":163,"threads_started":1}
I20260808 06:08:41.298956 32031 heartbeater.cc:347] Connected to a master server at 127.1.151.190:39173
I20260808 06:08:41.299189 32031 heartbeater.cc:464] Registering TS with master...
I20260808 06:08:41.299659 32031 heartbeater.cc:511] Master 127.1.151.190:39173 requested a full tablet report, sending...
I20260808 06:08:41.300598 31315 ts_manager.cc:194] Registered new tserver with Master: 0984588df55342b99f705767aba1c182 (127.1.151.131:40719)
I20260808 06:08:41.301129 31315 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.151.131:33593
I20260808 06:08:41.302832 1630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 06:08:41.308084 31385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:41.304725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59432 (local address 127.1.151.190:39173)
0808 06:08:41.304837 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:41.304841 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:41.304979 (+ 138us) server_negotiation.cc:407] Connection header received
0808 06:08:41.305069 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:41.305071 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:41.305107 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:41.305170 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:41.305354 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:41.305861 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:41.306713 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:41.306918 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:41.306991 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:41.307707 (+ 716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:41.307710 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:41.307712 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:08:41.307722 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:41.307738 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:08:41.307746 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:41.307835 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:41.307922 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 06:08:41.307954 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 06:08:41.310688 31315 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59432:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 06:08:41.313516 31315 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 06:08:41.327486 32050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:41.321285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:42213 (local address 127.0.0.1:47234)
0808 06:08:41.322064 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:41.322113 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:41.322256 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:41.323448 (+ 1192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:41.323450 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:41.323463 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:08:41.323534 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:41.323537 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:41.324812 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:41.324815 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:41.325928 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:41.325939 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:41.326970 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:41.326973 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:41.327082 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:41.327132 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 06:08:41.327235 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 06:08:41.327340 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"mutex_wait_us":39,"thread_start_us":60,"threads_started":1}
I20260808 06:08:41.327526 32057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:41.322409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47234 (local address 127.1.151.129:42213)
0808 06:08:41.323026 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:41.323038 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:41.323062 (+ 24us) server_negotiation.cc:407] Connection header received
0808 06:08:41.323139 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:41.323144 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:41.323328 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:41.323480 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:08:41.323613 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:41.324697 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:41.326081 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:41.326841 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:41.326949 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:41.327149 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:41.327266 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 06:08:41.327363 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":153,"threads_started":1}
I20260808 06:08:41.327759 32052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:41.321615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43870 (local address 127.1.151.130:37479)
0808 06:08:41.321915 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:41.321927 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:41.322416 (+ 489us) server_negotiation.cc:407] Connection header received
0808 06:08:41.322505 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:41.322511 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:41.322723 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:41.322854 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:08:41.323197 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:41.324408 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:41.326028 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:41.326842 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:41.326949 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:41.327137 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:41.327488 (+ 351us) server_negotiation.cc:299] Negotiation successful
0808 06:08:41.327593 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":99,"threads_started":1}
I20260808 06:08:41.328162 32053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:41.321401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:37479 (local address 127.0.0.1:43870)
0808 06:08:41.322227 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:41.322240 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:41.322339 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:41.322873 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:41.322879 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:41.322905 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:08:41.323036 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:41.323043 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:41.324546 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:41.324549 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:41.325908 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:41.325917 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:41.327218 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:41.327222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:41.327327 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:41.327350 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 06:08:41.327955 (+ 605us) client_negotiation.cc:240] Negotiation successful
0808 06:08:41.328035 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":727,"mutex_wait_us":29,"thread_start_us":92,"threads_started":1}
I20260808 06:08:41.329349 32051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:41.321966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:40719 (local address 127.0.0.1:35326)
0808 06:08:41.323099 (+ 1133us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:41.323112 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:41.323193 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:41.325499 (+ 2306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:41.325501 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:41.325534 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:08:41.325614 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:41.325620 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:41.326719 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:41.326723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:41.327920 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:41.327931 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:41.328988 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:41.328992 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:41.329084 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:41.329105 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 06:08:41.329165 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 06:08:41.329228 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1067,"thread_start_us":82,"threads_started":1}
I20260808 06:08:41.329452 32060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:41.322939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35326 (local address 127.1.151.131:40719)
0808 06:08:41.324965 (+ 2026us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:41.324976 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:41.324997 (+ 21us) server_negotiation.cc:407] Connection header received
0808 06:08:41.325065 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:41.325070 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:41.325238 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:41.325370 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:08:41.325742 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:41.326608 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:41.328150 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:41.328876 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:41.328972 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:41.329163 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:41.329210 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 06:08:41.329294 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1927,"thread_start_us":152,"threads_started":1}
I20260808 06:08:41.330840 31725 tablet_service.cc:1467] Processing CreateTablet for tablet 4b7fdb15fb9b4061995939c3479e1cf6 (DEFAULT_TABLE table=test-table [id=555cc8f6949f48fdb4cf1c37e7cc2ecd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:08:41.330842 31521 tablet_service.cc:1467] Processing CreateTablet for tablet 4b7fdb15fb9b4061995939c3479e1cf6 (DEFAULT_TABLE table=test-table [id=555cc8f6949f48fdb4cf1c37e7cc2ecd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:08:41.331477 31924 tablet_service.cc:1467] Processing CreateTablet for tablet 4b7fdb15fb9b4061995939c3479e1cf6 (DEFAULT_TABLE table=test-table [id=555cc8f6949f48fdb4cf1c37e7cc2ecd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:08:41.331982 31725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b7fdb15fb9b4061995939c3479e1cf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:08:41.331982 31521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b7fdb15fb9b4061995939c3479e1cf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:08:41.332309 31924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b7fdb15fb9b4061995939c3479e1cf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:08:41.338200 32062 tablet_bootstrap.cc:492] T 4b7fdb15fb9b4061995939c3479e1cf6 P 0984588df55342b99f705767aba1c182: Bootstrap starting.
I20260808 06:08:41.340035 32065 tablet_bootstrap.cc:492] T 4b7fdb15fb9b4061995939c3479e1cf6 P be8b6618164f4c6584db75195ccbf714: Bootstrap starting.
I20260808 06:08:41.340042 32069 tablet_bootstrap.cc:492] T 4b7fdb15fb9b4061995939c3479e1cf6 P 8a250dadf3e74a4dabe8705d2547ea3b: Bootstrap starting.
I20260808 06:08:41.340794 32062 tablet_bootstrap.cc:654] T 4b7fdb15fb9b4061995939c3479e1cf6 P 0984588df55342b99f705767aba1c182: Neither blocks nor log segments found. Creating new log.
I20260808 06:08:41.341472 32062 log.cc:824] T 4b7fdb15fb9b4061995939c3479e1cf6 P 0984588df55342b99f705767aba1c182: Log is configured to *not* fsync() on all Append() calls
I20260808 06:08:41.342234 32065 tablet_bootstrap.cc:654] T 4b7fdb15fb9b4061995939c3479e1cf6 P be8b6618164f4c6584db75195ccbf714: Neither blocks nor log segments found. Creating new log.
I20260808 06:08:41.342234 32069 tablet_bootstrap.cc:654] T 4b7fdb15fb9b4061995939c3479e1cf6 P 8a250dadf3e74a4dabe8705d2547ea3b: Neither blocks nor log segments found. Creating new log.
I20260808 06:08:41.343065 32065 log.cc:824] T 4b7fdb15fb9b4061995939c3479e1cf6 P be8b6618164f4c6584db75195ccbf714: Log is configured to *not* fsync() on all Append() calls
I20260808 06:08:41.343065 32069 log.cc:824] T 4b7fdb15fb9b4061995939c3479e1cf6 P 8a250dadf3e74a4dabe8705d2547ea3b: Log is configured to *not* fsync() on all Append() calls
I20260808 06:08:41.343180 32062 tablet_bootstrap.cc:492] T 4b7fdb15fb9b4061995939c3479e1cf6 P 0984588df55342b99f705767aba1c182: No bootstrap required, opened a new log
I20260808 06:08:41.343360 32062 ts_tablet_manager.cc:1397] T 4b7fdb15fb9b4061995939c3479e1cf6 P 0984588df55342b99f705767aba1c182: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 06:08:41.344964 32065 tablet_bootstrap.cc:492] T 4b7fdb15fb9b4061995939c3479e1cf6 P be8b6618164f4c6584db75195ccbf714: No bootstrap required, opened a new log
I20260808 06:08:41.344964 32069 tablet_bootstrap.cc:492] T 4b7fdb15fb9b4061995939c3479e1cf6 P 8a250dadf3e74a4dabe8705d2547ea3b: No bootstrap required, opened a new log
I20260808 06:08:41.345151 32065 ts_tablet_manager.cc:1397] T 4b7fdb15fb9b4061995939c3479e1cf6 P be8b6618164f4c6584db75195ccbf714: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260808 06:08:41.345151 32069 ts_tablet_manager.cc:1397] T 4b7fdb15fb9b4061995939c3479e1cf6 P 8a250dadf3e74a4dabe8705d2547ea3b: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260808 06:08:41.346798 32062 raft_consensus.cc:348] T 4b7fdb15fb9b4061995939c3479e1cf6 P 0984588df55342b99f705767aba1c182 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a250dadf3e74a4dabe8705d2547ea3b" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 42213 } } peers { permanent_uuid: "be8b6618164f4c6584db75195ccbf714" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 37479 } } peers { permanent_uuid: "0984588df55342b99f705767aba1c182" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 40719 } }
I20260808 06:08:41.347007 32062 raft_consensus.cc:374] T 4b7fdb15fb9b4061995939c3479e1cf6 P 0984588df55342b99f705767aba1c182 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:08:41.347059 32062 raft_consensus.cc:729] T 4b7fdb15fb9b4061995939c3479e1cf6 P 0984588df55342b99f705767aba1c182 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0984588df55342b99f705767aba1c182, State: Initialized, Role: FOLLOWER
I20260808 06:08:41.347445 32062 consensus_queue.cc:260] T 4b7fdb15fb9b4061995939c3479e1cf6 P 0984588df55342b99f705767aba1c182 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a250dadf3e74a4dabe8705d2547ea3b" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 42213 } } peers { permanent_uuid: "be8b6618164f4c6584db75195ccbf714" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 37479 } } peers { permanent_uuid: "0984588df55342b99f705767aba1c182" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 40719 } }
I20260808 06:08:41.348024 32031 heartbeater.cc:503] Master 127.1.151.190:39173 was elected leader, sending a full tablet report...
I20260808 06:08:41.348379 32062 ts_tablet_manager.cc:1428] T 4b7fdb15fb9b4061995939c3479e1cf6 P 0984588df55342b99f705767aba1c182: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 06:08:41.349308 32065 raft_consensus.cc:348] T 4b7fdb15fb9b4061995939c3479e1cf6 P be8b6618164f4c6584db75195ccbf714 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a250dadf3e74a4dabe8705d2547ea3b" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 42213 } } peers { permanent_uuid: "be8b6618164f4c6584db75195ccbf714" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 37479 } } peers { permanent_uuid: "0984588df55342b99f705767aba1c182" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 40719 } }
I20260808 06:08:41.349308 32069 raft_consensus.cc:348] T 4b7fdb15fb9b4061995939c3479e1cf6 P 8a250dadf3e74a4dabe8705d2547ea3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a250dadf3e74a4dabe8705d2547ea3b" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 42213 } } peers { permanent_uuid: "be8b6618164f4c6584db75195ccbf714" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 37479 } } peers { permanent_uuid: "0984588df55342b99f705767aba1c182" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 40719 } }
I20260808 06:08:41.349560 32069 raft_consensus.cc:374] T 4b7fdb15fb9b4061995939c3479e1cf6 P 8a250dadf3e74a4dabe8705d2547ea3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:08:41.349560 32065 raft_consensus.cc:374] T 4b7fdb15fb9b4061995939c3479e1cf6 P be8b6618164f4c6584db75195ccbf714 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:08:41.349627 32065 raft_consensus.cc:729] T 4b7fdb15fb9b4061995939c3479e1cf6 P be8b6618164f4c6584db75195ccbf714 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be8b6618164f4c6584db75195ccbf714, State: Initialized, Role: FOLLOWER
I20260808 06:08:41.349627 32069 raft_consensus.cc:729] T 4b7fdb15fb9b4061995939c3479e1cf6 P 8a250dadf3e74a4dabe8705d2547ea3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a250dadf3e74a4dabe8705d2547ea3b, State: Initialized, Role: FOLLOWER
I20260808 06:08:41.350061 32069 consensus_queue.cc:260] T 4b7fdb15fb9b4061995939c3479e1cf6 P 8a250dadf3e74a4dabe8705d2547ea3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a250dadf3e74a4dabe8705d2547ea3b" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 42213 } } peers { permanent_uuid: "be8b6618164f4c6584db75195ccbf714" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 37479 } } peers { permanent_uuid: "0984588df55342b99f705767aba1c182" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 40719 } }
I20260808 06:08:41.350060 32065 consensus_queue.cc:260] T 4b7fdb15fb9b4061995939c3479e1cf6 P be8b6618164f4c6584db75195ccbf714 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a250dadf3e74a4dabe8705d2547ea3b" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 42213 } } peers { permanent_uuid: "be8b6618164f4c6584db75195ccbf714" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 37479 } } peers { permanent_uuid: "0984588df55342b99f705767aba1c182" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 40719 } }
I20260808 06:08:41.350983 31615 heartbeater.cc:503] Master 127.1.151.190:39173 was elected leader, sending a full tablet report...
I20260808 06:08:41.351029 32065 ts_tablet_manager.cc:1428] T 4b7fdb15fb9b4061995939c3479e1cf6 P be8b6618164f4c6584db75195ccbf714: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 06:08:41.351020 32069 ts_tablet_manager.cc:1428] T 4b7fdb15fb9b4061995939c3479e1cf6 P 8a250dadf3e74a4dabe8705d2547ea3b: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 06:08:41.351531 31823 heartbeater.cc:503] Master 127.1.151.190:39173 was elected leader, sending a full tablet report...
W20260808 06:08:41.354753 31824 tablet.cc:2369] T 4b7fdb15fb9b4061995939c3479e1cf6 P be8b6618164f4c6584db75195ccbf714: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 06:08:41.407467 31616 tablet.cc:2369] T 4b7fdb15fb9b4061995939c3479e1cf6 P 8a250dadf3e74a4dabe8705d2547ea3b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 06:08:41.538919 32032 tablet.cc:2369] T 4b7fdb15fb9b4061995939c3479e1cf6 P 0984588df55342b99f705767aba1c182: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 06:08:41.544895 32075 raft_consensus.cc:482] T 4b7fdb15fb9b4061995939c3479e1cf6 P be8b6618164f4c6584db75195ccbf714 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:08:41.545107 32075 raft_consensus.cc:504] T 4b7fdb15fb9b4061995939c3479e1cf6 P be8b6618164f4c6584db75195ccbf714 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a250dadf3e74a4dabe8705d2547ea3b" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 42213 } } peers { permanent_uuid: "be8b6618164f4c6584db75195ccbf714" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 37479 } } peers { permanent_uuid: "0984588df55342b99f705767aba1c182" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 40719 } }
I20260808 06:08:41.546336 32075 leader_election.cc:288] T 4b7fdb15fb9b4061995939c3479e1cf6 P be8b6618164f4c6584db75195ccbf714 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a250dadf3e74a4dabe8705d2547ea3b (127.1.151.129:42213), 0984588df55342b99f705767aba1c182 (127.1.151.131:40719)
I20260808 06:08:41.550841 32057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:41.546713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:32861 (local address 127.1.151.129:42213)
0808 06:08:41.546845 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:41.546850 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:41.546869 (+ 19us) server_negotiation.cc:407] Connection header received
0808 06:08:41.546937 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:41.546940 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:41.546993 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:41.547082 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:08:41.547435 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:41.548178 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:41.549670 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:41.550259 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:41.550378 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:41.550431 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:41.550652 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 06:08:41.550717 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 06:08:41.550841 31827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:41.546582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:42213 (local address 127.1.151.130:32861)
0808 06:08:41.546714 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:41.546730 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:41.546832 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:41.547177 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:41.547180 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:41.547201 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:08:41.547326 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:41.547333 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:41.548291 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:41.548294 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:41.549540 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:41.549551 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:41.550413 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:41.550416 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:41.550511 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:41.550540 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 06:08:41.550622 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 06:08:41.550700 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 06:08:41.550978 32060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:41.547014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:52023 (local address 127.1.151.131:40719)
0808 06:08:41.547137 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:41.547141 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:41.547313 (+ 172us) server_negotiation.cc:407] Connection header received
0808 06:08:41.547415 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:41.547418 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:41.547466 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:41.547537 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:08:41.547783 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:41.548429 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:41.549828 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:41.550429 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:41.550529 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:41.550564 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:41.550802 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 06:08:41.550872 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 06:08:41.551002 32077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:41.546871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:40719 (local address 127.1.151.130:52023)
0808 06:08:41.547228 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:41.547240 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:41.547330 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:41.547566 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:41.547568 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:41.547584 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:08:41.547691 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:41.547695 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:41.548524 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:41.548527 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:41.549660 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:41.549668 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:41.550557 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:41.550561 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:41.550685 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:41.550705 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 06:08:41.550775 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 06:08:41.550827 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":123,"threads_started":1}
I20260808 06:08:41.551585 31554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b7fdb15fb9b4061995939c3479e1cf6" candidate_uuid: "be8b6618164f4c6584db75195ccbf714" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a250dadf3e74a4dabe8705d2547ea3b" is_pre_election: true
I20260808 06:08:41.551591 31960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b7fdb15fb9b4061995939c3479e1cf6" candidate_uuid: "be8b6618164f4c6584db75195ccbf714" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0984588df55342b99f705767aba1c182" is_pre_election: true
I20260808 06:08:41.551865 31554 raft_consensus.cc:2454] T 4b7fdb15fb9b4061995939c3479e1cf6 P 8a250dadf3e74a4dabe8705d2547ea3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be8b6618164f4c6584db75195ccbf714 in term 0.
I20260808 06:08:41.551872 31960 raft_consensus.cc:2454] T 4b7fdb15fb9b4061995939c3479e1cf6 P 0984588df55342b99f705767aba1c182 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be8b6618164f4c6584db75195ccbf714 in term 0.
I20260808 06:08:41.552315 31656 leader_election.cc:302] T 4b7fdb15fb9b4061995939c3479e1cf6 P be8b6618164f4c6584db75195ccbf714 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a250dadf3e74a4dabe8705d2547ea3b, be8b6618164f4c6584db75195ccbf714; no voters:
I20260808 06:08:41.552587 32075 raft_consensus.cc:2781] T 4b7fdb15fb9b4061995939c3479e1cf6 P be8b6618164f4c6584db75195ccbf714 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 06:08:41.552698 32075 raft_consensus.cc:482] T 4b7fdb15fb9b4061995939c3479e1cf6 P be8b6618164f4c6584db75195ccbf714 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 06:08:41.552731 32075 raft_consensus.cc:3037] T 4b7fdb15fb9b4061995939c3479e1cf6 P be8b6618164f4c6584db75195ccbf714 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:08:41.553839 32075 raft_consensus.cc:504] T 4b7fdb15fb9b4061995939c3479e1cf6 P be8b6618164f4c6584db75195ccbf714 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a250dadf3e74a4dabe8705d2547ea3b" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 42213 } } peers { permanent_uuid: "be8b6618164f4c6584db75195ccbf714" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 37479 } } peers { permanent_uuid: "0984588df55342b99f705767aba1c182" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 40719 } }
I20260808 06:08:41.554205 32075 leader_election.cc:288] T 4b7fdb15fb9b4061995939c3479e1cf6 P be8b6618164f4c6584db75195ccbf714 [CANDIDATE]: Term 1 election: Requested vote from peers 8a250dadf3e74a4dabe8705d2547ea3b (127.1.151.129:42213), 0984588df55342b99f705767aba1c182 (127.1.151.131:40719)
I20260808 06:08:41.554517 31554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b7fdb15fb9b4061995939c3479e1cf6" candidate_uuid: "be8b6618164f4c6584db75195ccbf714" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a250dadf3e74a4dabe8705d2547ea3b"
I20260808 06:08:41.554616 31960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b7fdb15fb9b4061995939c3479e1cf6" candidate_uuid: "be8b6618164f4c6584db75195ccbf714" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0984588df55342b99f705767aba1c182"
I20260808 06:08:41.554646 31554 raft_consensus.cc:3037] T 4b7fdb15fb9b4061995939c3479e1cf6 P 8a250dadf3e74a4dabe8705d2547ea3b [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:08:41.554728 31960 raft_consensus.cc:3037] T 4b7fdb15fb9b4061995939c3479e1cf6 P 0984588df55342b99f705767aba1c182 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:08:41.555881 31554 raft_consensus.cc:2454] T 4b7fdb15fb9b4061995939c3479e1cf6 P 8a250dadf3e74a4dabe8705d2547ea3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be8b6618164f4c6584db75195ccbf714 in term 1.
I20260808 06:08:41.555883 31960 raft_consensus.cc:2454] T 4b7fdb15fb9b4061995939c3479e1cf6 P 0984588df55342b99f705767aba1c182 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be8b6618164f4c6584db75195ccbf714 in term 1.
I20260808 06:08:41.556185 31656 leader_election.cc:302] T 4b7fdb15fb9b4061995939c3479e1cf6 P be8b6618164f4c6584db75195ccbf714 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0984588df55342b99f705767aba1c182, be8b6618164f4c6584db75195ccbf714; no voters:
I20260808 06:08:41.556360 32075 raft_consensus.cc:2781] T 4b7fdb15fb9b4061995939c3479e1cf6 P be8b6618164f4c6584db75195ccbf714 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:08:41.556569 32075 raft_consensus.cc:686] T 4b7fdb15fb9b4061995939c3479e1cf6 P be8b6618164f4c6584db75195ccbf714 [term 1 LEADER]: Becoming Leader. State: Replica: be8b6618164f4c6584db75195ccbf714, State: Running, Role: LEADER
I20260808 06:08:41.556869 32075 consensus_queue.cc:237] T 4b7fdb15fb9b4061995939c3479e1cf6 P be8b6618164f4c6584db75195ccbf714 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a250dadf3e74a4dabe8705d2547ea3b" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 42213 } } peers { permanent_uuid: "be8b6618164f4c6584db75195ccbf714" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 37479 } } peers { permanent_uuid: "0984588df55342b99f705767aba1c182" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 40719 } }
I20260808 06:08:41.559756 31314 catalog_manager.cc:5495] T 4b7fdb15fb9b4061995939c3479e1cf6 P be8b6618164f4c6584db75195ccbf714 reported cstate change: term changed from 0 to 1, leader changed from <none> to be8b6618164f4c6584db75195ccbf714 (127.1.151.130). New cstate: current_term: 1 leader_uuid: "be8b6618164f4c6584db75195ccbf714" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a250dadf3e74a4dabe8705d2547ea3b" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 42213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be8b6618164f4c6584db75195ccbf714" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 37479 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0984588df55342b99f705767aba1c182" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 40719 } health_report { overall_health: UNKNOWN } } }
I20260808 06:08:41.619695 32052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:41.615961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43872 (local address 127.1.151.130:37479)
0808 06:08:41.616117 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:41.616122 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:41.616138 (+ 16us) server_negotiation.cc:407] Connection header received
0808 06:08:41.616226 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:41.616228 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:41.616287 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:41.616381 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:08:41.616707 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:41.617235 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:41.618147 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:41.618403 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:41.618491 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:41.618851 (+ 360us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:08:41.619256 (+ 405us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:08:41.619364 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:41.619503 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 06:08:41.619538 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 06:08:41.625483 31960 raft_consensus.cc:1261] T 4b7fdb15fb9b4061995939c3479e1cf6 P 0984588df55342b99f705767aba1c182 [term 1 FOLLOWER]: Refusing update from remote peer be8b6618164f4c6584db75195ccbf714: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:08:41.625483 31554 raft_consensus.cc:1261] T 4b7fdb15fb9b4061995939c3479e1cf6 P 8a250dadf3e74a4dabe8705d2547ea3b [term 1 FOLLOWER]: Refusing update from remote peer be8b6618164f4c6584db75195ccbf714: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:08:41.626202 32080 consensus_queue.cc:1035] T 4b7fdb15fb9b4061995939c3479e1cf6 P be8b6618164f4c6584db75195ccbf714 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a250dadf3e74a4dabe8705d2547ea3b" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 42213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:08:41.626384 32075 consensus_queue.cc:1035] T 4b7fdb15fb9b4061995939c3479e1cf6 P be8b6618164f4c6584db75195ccbf714 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0984588df55342b99f705767aba1c182" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 40719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:08:41.634665 32087 mvcc.cc:205] Tried to move back new op lower bound from 7316149541370925056 to 7316149541099315200. Current Snapshot: MvccSnapshot[applied={T|T < 7316149541370925056}]
I20260808 06:08:41.640735 31314 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:08:41.640898 31314 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:08:41.643270 31314 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6d7835e3098f4fda8ca87ab710985cf6: Sending DeleteTablet for 3 replicas of tablet 4b7fdb15fb9b4061995939c3479e1cf6
I20260808 06:08:41.643985 31725 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b7fdb15fb9b4061995939c3479e1cf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:08:41 UTC) from {username='slave'} at 127.0.0.1:43870
I20260808 06:08:41.644047 31521 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b7fdb15fb9b4061995939c3479e1cf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:08:41 UTC) from {username='slave'} at 127.0.0.1:47234
I20260808 06:08:41.644044 31924 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b7fdb15fb9b4061995939c3479e1cf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:08:41 UTC) from {username='slave'} at 127.0.0.1:35326
I20260808 06:08:41.644450 32103 tablet_replica.cc:330] T 4b7fdb15fb9b4061995939c3479e1cf6 P 0984588df55342b99f705767aba1c182: stopping tablet replica
I20260808 06:08:41.644619 32104 tablet_replica.cc:330] T 4b7fdb15fb9b4061995939c3479e1cf6 P be8b6618164f4c6584db75195ccbf714: stopping tablet replica
I20260808 06:08:41.644853 32104 raft_consensus.cc:2229] T 4b7fdb15fb9b4061995939c3479e1cf6 P be8b6618164f4c6584db75195ccbf714 [term 1 LEADER]: Raft consensus shutting down.
I20260808 06:08:41.644853 32103 raft_consensus.cc:2229] T 4b7fdb15fb9b4061995939c3479e1cf6 P 0984588df55342b99f705767aba1c182 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:08:41.645148 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 31419
I20260808 06:08:41.645171 32104 raft_consensus.cc:2258] T 4b7fdb15fb9b4061995939c3479e1cf6 P be8b6618164f4c6584db75195ccbf714 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:08:41.645434 32103 raft_consensus.cc:2258] T 4b7fdb15fb9b4061995939c3479e1cf6 P 0984588df55342b99f705767aba1c182 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:08:41.646651 32104 ts_tablet_manager.cc:1905] T 4b7fdb15fb9b4061995939c3479e1cf6 P be8b6618164f4c6584db75195ccbf714: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:08:41.646927 32103 ts_tablet_manager.cc:1905] T 4b7fdb15fb9b4061995939c3479e1cf6 P 0984588df55342b99f705767aba1c182: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:08:41.649370 32104 ts_tablet_manager.cc:1918] T 4b7fdb15fb9b4061995939c3479e1cf6 P be8b6618164f4c6584db75195ccbf714: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:08:41.649474 32104 log.cc:1196] T 4b7fdb15fb9b4061995939c3479e1cf6 P be8b6618164f4c6584db75195ccbf714: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-1/wal/wals/4b7fdb15fb9b4061995939c3479e1cf6
I20260808 06:08:41.649745 32104 ts_tablet_manager.cc:1939] T 4b7fdb15fb9b4061995939c3479e1cf6 P be8b6618164f4c6584db75195ccbf714: Deleting consensus metadata
I20260808 06:08:41.650408 32103 ts_tablet_manager.cc:1918] T 4b7fdb15fb9b4061995939c3479e1cf6 P 0984588df55342b99f705767aba1c182: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:08:41.650545 32103 log.cc:1196] T 4b7fdb15fb9b4061995939c3479e1cf6 P 0984588df55342b99f705767aba1c182: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786169070948234-1630-0/minicluster-data/ts-2/wal/wals/4b7fdb15fb9b4061995939c3479e1cf6
I20260808 06:08:41.650929 32103 ts_tablet_manager.cc:1939] T 4b7fdb15fb9b4061995939c3479e1cf6 P 0984588df55342b99f705767aba1c182: Deleting consensus metadata
I20260808 06:08:41.652412 31299 catalog_manager.cc:4841] TS 0984588df55342b99f705767aba1c182 (127.1.151.131:40719): tablet 4b7fdb15fb9b4061995939c3479e1cf6 (table test-table [id=555cc8f6949f48fdb4cf1c37e7cc2ecd]) successfully deleted
I20260808 06:08:41.653017 31297 catalog_manager.cc:4841] TS be8b6618164f4c6584db75195ccbf714 (127.1.151.130:37479): tablet 4b7fdb15fb9b4061995939c3479e1cf6 (table test-table [id=555cc8f6949f48fdb4cf1c37e7cc2ecd]) successfully deleted
W20260808 06:08:41.662751 31299 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.151.129:42213 (error 108)
I20260808 06:08:41.662879 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 31618
I20260808 06:08:41.667949 32051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:41.667368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:42213 (local address 127.0.0.1:47242)
0808 06:08:41.667758 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:41.667842 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.129:42213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":290}
W20260808 06:08:41.671679 31299 catalog_manager.cc:4584] TS 8a250dadf3e74a4dabe8705d2547ea3b (127.1.151.129:42213): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4b7fdb15fb9b4061995939c3479e1cf6: Network error: Client connection negotiation failed: client connection to 127.1.151.129:42213: connect: Connection refused (error 111)
I20260808 06:08:41.676563 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 31828
I20260808 06:08:41.690503 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 31270
2026-08-08T06:08:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1233 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:08:41 dist-test-slave-zj8q krb5kdc[32108](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:08:41 dist-test-slave-zj8q krb5kdc[32108](info): set up 2 sockets
Aug 08 06:08:41 dist-test-slave-zj8q krb5kdc[32108](info): commencing operation
krb5kdc: starting...
W20260808 06:08:42.837088 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:08:42 dist-test-slave-zj8q krb5kdc[32108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:08:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:08:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.151.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
I20260808 06:08:42.952806 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:46459
--webserver_interface=127.1.151.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:43179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:46459
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:08:43.062086 32130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:43.062330 32130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:43.062372 32130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:43.066274 32130 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:08:43.066326 32130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:43.066344 32130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:43.066361 32130 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:08:43.066375 32130 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:08:43.071424 32130 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:43179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:46459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:46459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:43.072388 32130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:43.073325 32130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:08:43.081537 32137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:43.081563 32136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:43.081647 32130 server_base.cc:1034] running on GCE node
W20260808 06:08:43.081596 32140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:43.082362 32130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:43.083413 32130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:43.084677 32130 hybrid_clock.cc:648] HybridClock initialized: now 1786169323084659 us; error 29 us; skew 500 ppm
Aug 08 06:08:43 dist-test-slave-zj8q krb5kdc[32108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169323, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:08:43.087576 32130 init.cc:377] Logged in from keytab as kudu/127.1.151.190@KRBTEST.COM (short username kudu)
I20260808 06:08:43.088577 32130 webserver.cc:460] Webserver started at http://127.1.151.190:40321/ using document root <none> and password file <none>
I20260808 06:08:43.089166 32130 fs_manager.cc:359] Metadata directory not provided
I20260808 06:08:43.089236 32130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:08:43.089433 32130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:08:43.091569 32130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "28d25033676445518e2eaea1920f88f6"
format_stamp: "Formatted at 2026-08-08 06:08:43 on dist-test-slave-zj8q"
server_key: "8c0c68f446acea72471eb60811ee9720"
server_key_iv: "f6bb6f2512334d9a36f44db78d41e2c2"
server_key_version: "encryptionkey@0"
I20260808 06:08:43.092168 32130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "28d25033676445518e2eaea1920f88f6"
format_stamp: "Formatted at 2026-08-08 06:08:43 on dist-test-slave-zj8q"
server_key: "8c0c68f446acea72471eb60811ee9720"
server_key_iv: "f6bb6f2512334d9a36f44db78d41e2c2"
server_key_version: "encryptionkey@0"
I20260808 06:08:43.095950 32130 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 06:08:43.098354 32147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:43.099536 32130 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:08:43.099676 32130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "28d25033676445518e2eaea1920f88f6"
format_stamp: "Formatted at 2026-08-08 06:08:43 on dist-test-slave-zj8q"
server_key: "8c0c68f446acea72471eb60811ee9720"
server_key_iv: "f6bb6f2512334d9a36f44db78d41e2c2"
server_key_version: "encryptionkey@0"
I20260808 06:08:43.099779 32130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:08:43.109687 32130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:08:43.112869 32130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:08:43.113049 32130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:08:43.119915 32130 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:46459
I20260808 06:08:43.119951 32219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:46459 every 8 connection(s)
I20260808 06:08:43.120862 32130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:08:43.123535 32220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:08:43.128671 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 32130
I20260808 06:08:43.128785 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:08:43.129043 1630 external_mini_cluster.cc:1428] Setting key a62642de6c86c0586d349c223bc4bd0a
I20260808 06:08:43.129195 32220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28d25033676445518e2eaea1920f88f6: Bootstrap starting.
I20260808 06:08:43.132117 32220 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28d25033676445518e2eaea1920f88f6: Neither blocks nor log segments found. Creating new log.
I20260808 06:08:43.133045 32220 log.cc:824] T 00000000000000000000000000000000 P 28d25033676445518e2eaea1920f88f6: Log is configured to *not* fsync() on all Append() calls
I20260808 06:08:43.134972 32220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28d25033676445518e2eaea1920f88f6: No bootstrap required, opened a new log
Aug 08 06:08:43 dist-test-slave-zj8q krb5kdc[32108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:08:43.138841 32220 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28d25033676445518e2eaea1920f88f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d25033676445518e2eaea1920f88f6" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 46459 } }
I20260808 06:08:43.139122 32220 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28d25033676445518e2eaea1920f88f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:08:43.139182 32220 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28d25033676445518e2eaea1920f88f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28d25033676445518e2eaea1920f88f6, State: Initialized, Role: FOLLOWER
I20260808 06:08:43.139633 32220 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28d25033676445518e2eaea1920f88f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d25033676445518e2eaea1920f88f6" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 46459 } }
I20260808 06:08:43.139767 32220 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28d25033676445518e2eaea1920f88f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:08:43.139838 32220 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28d25033676445518e2eaea1920f88f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:08:43.139933 32220 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28d25033676445518e2eaea1920f88f6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:08:43.140795 32220 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28d25033676445518e2eaea1920f88f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d25033676445518e2eaea1920f88f6" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 46459 } }
I20260808 06:08:43.141139 32220 leader_election.cc:302] T 00000000000000000000000000000000 P 28d25033676445518e2eaea1920f88f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28d25033676445518e2eaea1920f88f6; no voters:
I20260808 06:08:43.141428 32220 leader_election.cc:288] T 00000000000000000000000000000000 P 28d25033676445518e2eaea1920f88f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:08:43.141512 32227 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28d25033676445518e2eaea1920f88f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:08:43.141721 32227 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28d25033676445518e2eaea1920f88f6 [term 1 LEADER]: Becoming Leader. State: Replica: 28d25033676445518e2eaea1920f88f6, State: Running, Role: LEADER
I20260808 06:08:43.142030 32227 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28d25033676445518e2eaea1920f88f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d25033676445518e2eaea1920f88f6" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 46459 } }
I20260808 06:08:43.142829 32220 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28d25033676445518e2eaea1920f88f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:08:43.143435 32228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28d25033676445518e2eaea1920f88f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28d25033676445518e2eaea1920f88f6. Latest consensus state: current_term: 1 leader_uuid: "28d25033676445518e2eaea1920f88f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d25033676445518e2eaea1920f88f6" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 46459 } } }
I20260808 06:08:43.143409 32229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28d25033676445518e2eaea1920f88f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28d25033676445518e2eaea1920f88f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d25033676445518e2eaea1920f88f6" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 46459 } } }
I20260808 06:08:43.143581 32228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28d25033676445518e2eaea1920f88f6 [sys.catalog]: This master's current role is: LEADER
I20260808 06:08:43.143599 32229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28d25033676445518e2eaea1920f88f6 [sys.catalog]: This master's current role is: LEADER
I20260808 06:08:43.143402 32224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:43.130419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35736 (local address 127.1.151.190:46459)
0808 06:08:43.130854 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:43.130873 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:43.130918 (+ 45us) server_negotiation.cc:407] Connection header received
0808 06:08:43.131750 (+ 832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:43.131775 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:43.132215 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:43.132600 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:43.133282 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.134354 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.135221 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.135695 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.135842 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:43.138478 (+ 2636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:43.138484 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:43.138496 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:08:43.138522 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:43.140308 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:43.140708 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:43.140714 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:43.140719 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:43.140831 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:43.141128 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:43.141131 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:43.141133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:43.141632 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:43.141814 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:43.142183 (+ 369us) server_negotiation.cc:299] Negotiation successful
0808 06:08:43.142500 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":123,"threads_started":1}
I20260808 06:08:43.144222 32235 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:08:43.147405 32235 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:08:43.160002 32235 catalog_manager.cc:1347] Generated new cluster ID: e73e776418514724aa2928ab768279f6
I20260808 06:08:43.160171 32235 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:08:43.169144 32235 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:08:43.169996 32235 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:08:43.180393 32235 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28d25033676445518e2eaea1920f88f6: Generated new TSK 0
I20260808 06:08:43.181012 32235 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.151.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab.
Entry for principal kudu/127.1.151.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab.
I20260808 06:08:43.208722 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.129:0
--local_ip_for_outbound_sockets=127.1.151.129
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:46459
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:43179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:08:43.328022 32260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:43.328270 32260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:43.328332 32260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:43.332340 32260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:43.332399 32260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:43.332477 32260 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.129
I20260808 06:08:43.337741 32260 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:43179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.129.keytab
--principal=kudu/127.1.151.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:46459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.129
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:43.338830 32260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:43.339838 32260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:08:43.350648 32269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:43.350726 32271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:43.350962 32268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:43.351770 32260 server_base.cc:1034] running on GCE node
I20260808 06:08:43.352264 32260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:43.353075 32260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:43.354231 32260 hybrid_clock.cc:648] HybridClock initialized: now 1786169323354204 us; error 39 us; skew 500 ppm
Aug 08 06:08:43 dist-test-slave-zj8q krb5kdc[32108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169323, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:08:43.357736 32260 init.cc:377] Logged in from keytab as kudu/127.1.151.129@KRBTEST.COM (short username kudu)
I20260808 06:08:43.358906 32260 webserver.cc:460] Webserver started at http://127.1.151.129:34827/ using document root <none> and password file <none>
I20260808 06:08:43.359666 32260 fs_manager.cc:359] Metadata directory not provided
I20260808 06:08:43.359772 32260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:08:43.360021 32260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:08:43.362951 32260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-0/data/instance:
uuid: "586e858ffc3f4c7baf9c27714e48d76b"
format_stamp: "Formatted at 2026-08-08 06:08:43 on dist-test-slave-zj8q"
server_key: "48142c3a59013f99c4ea7f98b9c673cb"
server_key_iv: "83bfade874e1496631946af6c8671155"
server_key_version: "encryptionkey@0"
I20260808 06:08:43.363686 32260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance:
uuid: "586e858ffc3f4c7baf9c27714e48d76b"
format_stamp: "Formatted at 2026-08-08 06:08:43 on dist-test-slave-zj8q"
server_key: "48142c3a59013f99c4ea7f98b9c673cb"
server_key_iv: "83bfade874e1496631946af6c8671155"
server_key_version: "encryptionkey@0"
I20260808 06:08:43.368676 32260 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 06:08:43.371608 32282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:43.372895 32260 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 06:08:43.373008 32260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-0/wal
uuid: "586e858ffc3f4c7baf9c27714e48d76b"
format_stamp: "Formatted at 2026-08-08 06:08:43 on dist-test-slave-zj8q"
server_key: "48142c3a59013f99c4ea7f98b9c673cb"
server_key_iv: "83bfade874e1496631946af6c8671155"
server_key_version: "encryptionkey@0"
I20260808 06:08:43.373111 32260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:08:43.386004 32260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:08:43.388955 32260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:08:43.389118 32260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:08:43.389670 32260 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:08:43.390538 32260 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:08:43.390586 32260 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:43.390646 32260 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:08:43.390669 32260 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:43.400020 32260 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.129:37231
I20260808 06:08:43.400079 32479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.129:37231 every 8 connection(s)
I20260808 06:08:43.400888 32260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
I20260808 06:08:43.405364 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 32260
I20260808 06:08:43.405489 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance
I20260808 06:08:43.405829 1630 external_mini_cluster.cc:1428] Setting key 623e0610732b15b3eec055b293ec59e1
Aug 08 06:08:43 dist-test-slave-zj8q krb5kdc[32108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169323, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.129@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:08:43.415906 32224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:43.402715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:59025 (local address 127.1.151.190:46459)
0808 06:08:43.402844 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:43.402847 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:43.403834 (+ 987us) server_negotiation.cc:407] Connection header received
0808 06:08:43.405135 (+ 1301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:43.405138 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:43.405196 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:43.405273 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:43.406703 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.407447 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.408579 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.408817 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.408907 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:43.411741 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:43.411745 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:43.411748 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:08:43.411774 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:43.413437 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:43.414073 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:43.414077 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:43.414079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:43.414219 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:43.414620 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:43.414624 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:43.414626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:43.414855 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:43.414961 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:43.415590 (+ 629us) server_negotiation.cc:299] Negotiation successful
0808 06:08:43.415740 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 06:08:43.416899 32486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:43.403171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:46459 (local address 127.1.151.129:59025)
0808 06:08:43.403679 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:43.403720 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:43.404930 (+ 1210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:43.405551 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:43.405560 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:43.405935 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:08:43.406547 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:43.406569 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.407568 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.407572 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:43.408463 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:43.408473 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.408954 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.408959 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:43.409135 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:43.409676 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:08:43.409697 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:08:43.411561 (+ 1864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:08:43.413578 (+ 2017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:08:43.413584 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:08:43.413599 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:08:43.413961 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:08:43.414318 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:08:43.414322 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:08:43.414324 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:08:43.414512 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:08:43.414965 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:08:43.414972 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:08:43.415307 (+ 335us) client_negotiation.cc:769] Sending connection context
0808 06:08:43.415523 (+ 216us) client_negotiation.cc:240] Negotiation successful
0808 06:08:43.415852 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":140,"threads_started":1}
WARNING: no policy specified for kudu/127.1.151.130@KRBTEST.COM; defaulting to no policy
I20260808 06:08:43.418279 32482 heartbeater.cc:347] Connected to a master server at 127.1.151.190:46459
I20260808 06:08:43.418510 32482 heartbeater.cc:464] Registering TS with master...
I20260808 06:08:43.418970 32482 heartbeater.cc:511] Master 127.1.151.190:46459 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.130@KRBTEST.COM" created.
I20260808 06:08:43.420635 32172 ts_manager.cc:194] Registered new tserver with Master: 586e858ffc3f4c7baf9c27714e48d76b (127.1.151.129:37231)
I20260808 06:08:43.421694 32172 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.129@KRBTEST.COM'} at 127.1.151.129:59025
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab.
Entry for principal kudu/127.1.151.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab.
I20260808 06:08:43.436810 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.130:0
--local_ip_for_outbound_sockets=127.1.151.130
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:46459
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:43179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:08:43.547297 32490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:43.547519 32490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:43.547559 32490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:43.551474 32490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:43.551525 32490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:43.551584 32490 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.130
I20260808 06:08:43.556718 32490 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:43179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.130.keytab
--principal=kudu/127.1.151.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:46459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32490
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.130
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:43.557659 32490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:43.558636 32490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:08:43.567953 32500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:43.568120 32490 server_base.cc:1034] running on GCE node
W20260808 06:08:43.567955 32503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:43.567955 32499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:43.568601 32490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:43.569183 32490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:43.570336 32490 hybrid_clock.cc:648] HybridClock initialized: now 1786169323570323 us; error 25 us; skew 500 ppm
Aug 08 06:08:43 dist-test-slave-zj8q krb5kdc[32108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169323, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:08:43.572940 32490 init.cc:377] Logged in from keytab as kudu/127.1.151.130@KRBTEST.COM (short username kudu)
I20260808 06:08:43.573752 32490 webserver.cc:460] Webserver started at http://127.1.151.130:45885/ using document root <none> and password file <none>
I20260808 06:08:43.574358 32490 fs_manager.cc:359] Metadata directory not provided
I20260808 06:08:43.574433 32490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:08:43.574642 32490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:08:43.576611 32490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-1/data/instance:
uuid: "892f14f5be9c4191b000fe1151d8e54d"
format_stamp: "Formatted at 2026-08-08 06:08:43 on dist-test-slave-zj8q"
server_key: "8aa49acfc35131ebe938f8937eecb384"
server_key_iv: "b7e616775c6b7513925c65bb9bb0b68f"
server_key_version: "encryptionkey@0"
I20260808 06:08:43.577195 32490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance:
uuid: "892f14f5be9c4191b000fe1151d8e54d"
format_stamp: "Formatted at 2026-08-08 06:08:43 on dist-test-slave-zj8q"
server_key: "8aa49acfc35131ebe938f8937eecb384"
server_key_iv: "b7e616775c6b7513925c65bb9bb0b68f"
server_key_version: "encryptionkey@0"
I20260808 06:08:43.580753 32490 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260808 06:08:43.583096 32517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:43.584298 32490 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:08:43.584414 32490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-1/wal
uuid: "892f14f5be9c4191b000fe1151d8e54d"
format_stamp: "Formatted at 2026-08-08 06:08:43 on dist-test-slave-zj8q"
server_key: "8aa49acfc35131ebe938f8937eecb384"
server_key_iv: "b7e616775c6b7513925c65bb9bb0b68f"
server_key_version: "encryptionkey@0"
I20260808 06:08:43.584511 32490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:08:43.597146 32490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:08:43.599808 32490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:08:43.599972 32490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:08:43.600512 32490 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:08:43.601290 32490 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:08:43.601336 32490 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:43.601387 32490 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:08:43.601415 32490 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:43.609826 32490 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.130:34705
I20260808 06:08:43.609884 32701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.130:34705 every 8 connection(s)
I20260808 06:08:43.610683 32490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
I20260808 06:08:43.613574 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 32490
I20260808 06:08:43.613722 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance
I20260808 06:08:43.614076 1630 external_mini_cluster.cc:1428] Setting key a08eb0e5e97b1bc1c312d2b954c699ae
Aug 08 06:08:43 dist-test-slave-zj8q krb5kdc[32108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169323, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.130@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.151.131@KRBTEST.COM; defaulting to no policy
I20260808 06:08:43.624984 32224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:43.612416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:54933 (local address 127.1.151.190:46459)
0808 06:08:43.612553 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:43.612557 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:43.613328 (+ 771us) server_negotiation.cc:407] Connection header received
0808 06:08:43.615005 (+ 1677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:43.615009 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:43.615060 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:43.615143 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:43.616565 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.617269 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.618689 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.618915 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.619006 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:43.621526 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:43.621529 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:43.621531 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:08:43.621555 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:43.622883 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:43.623454 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:43.623457 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:43.623458 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:43.623528 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:43.623869 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:43.623872 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:43.623873 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:43.624042 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:43.624115 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:43.624716 (+ 601us) server_negotiation.cc:299] Negotiation successful
0808 06:08:43.624842 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 06:08:43.625803 32705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:43.612760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:46459 (local address 127.1.151.130:54933)
0808 06:08:43.613180 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:43.613228 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:43.614792 (+ 1564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:43.615304 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:43.615311 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:43.615786 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:08:43.616417 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:43.616436 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.617399 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.617403 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:43.618568 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:43.618578 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.619054 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.619058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:43.619231 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:43.619736 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:08:43.619753 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:08:43.621354 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:08:43.622998 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:08:43.623005 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:08:43.623020 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:08:43.623366 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:08:43.623608 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:08:43.623612 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:08:43.623614 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:08:43.623785 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:08:43.624150 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:08:43.624156 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:08:43.624453 (+ 297us) client_negotiation.cc:769] Sending connection context
0808 06:08:43.624646 (+ 193us) client_negotiation.cc:240] Negotiation successful
0808 06:08:43.624972 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":100,"threads_started":1}
I20260808 06:08:43.626906 32702 heartbeater.cc:347] Connected to a master server at 127.1.151.190:46459
I20260808 06:08:43.627115 32702 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.131@KRBTEST.COM" created.
I20260808 06:08:43.627601 32702 heartbeater.cc:511] Master 127.1.151.190:46459 requested a full tablet report, sending...
I20260808 06:08:43.628768 32172 ts_manager.cc:194] Registered new tserver with Master: 892f14f5be9c4191b000fe1151d8e54d (127.1.151.130:34705)
I20260808 06:08:43.629495 32172 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.130@KRBTEST.COM'} at 127.1.151.130:54933
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab.
Entry for principal kudu/127.1.151.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab.
I20260808 06:08:43.640743 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.131:0
--local_ip_for_outbound_sockets=127.1.151.131
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:46459
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:43179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:08:43.753422 32708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:43.753670 32708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:43.753711 32708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:43.757679 32708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:43.757733 32708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:43.757787 32708 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.131
I20260808 06:08:43.762952 32708 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:43179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.131.keytab
--principal=kudu/127.1.151.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:46459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.131
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:43.763855 32708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:43.765067 32708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:08:43.774395 32718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:43.774456 32708 server_base.cc:1034] running on GCE node
W20260808 06:08:43.774401 32724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:43.774401 32721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:43.774953 32708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:43.775588 32708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:43.776752 32708 hybrid_clock.cc:648] HybridClock initialized: now 1786169323776724 us; error 48 us; skew 500 ppm
Aug 08 06:08:43 dist-test-slave-zj8q krb5kdc[32108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169323, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:08:43.779500 32708 init.cc:377] Logged in from keytab as kudu/127.1.151.131@KRBTEST.COM (short username kudu)
I20260808 06:08:43.780325 32708 webserver.cc:460] Webserver started at http://127.1.151.131:38039/ using document root <none> and password file <none>
I20260808 06:08:43.780834 32708 fs_manager.cc:359] Metadata directory not provided
I20260808 06:08:43.780894 32708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:08:43.781090 32708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:08:43.783089 32708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-2/data/instance:
uuid: "2ab96b0c917347febf972bc30d1f5165"
format_stamp: "Formatted at 2026-08-08 06:08:43 on dist-test-slave-zj8q"
server_key: "1665d47f4789625775af8ce2df1e01e8"
server_key_iv: "343b55f3e7870c2570d1d0adb36156a9"
server_key_version: "encryptionkey@0"
I20260808 06:08:43.783623 32708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance:
uuid: "2ab96b0c917347febf972bc30d1f5165"
format_stamp: "Formatted at 2026-08-08 06:08:43 on dist-test-slave-zj8q"
server_key: "1665d47f4789625775af8ce2df1e01e8"
server_key_iv: "343b55f3e7870c2570d1d0adb36156a9"
server_key_version: "encryptionkey@0"
I20260808 06:08:43.787164 32708 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 06:08:43.789595 32734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:43.790899 32708 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:08:43.791002 32708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-2/wal
uuid: "2ab96b0c917347febf972bc30d1f5165"
format_stamp: "Formatted at 2026-08-08 06:08:43 on dist-test-slave-zj8q"
server_key: "1665d47f4789625775af8ce2df1e01e8"
server_key_iv: "343b55f3e7870c2570d1d0adb36156a9"
server_key_version: "encryptionkey@0"
I20260808 06:08:43.791107 32708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:08:43.805171 32708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:08:43.808123 32708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:08:43.808284 32708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:08:43.808821 32708 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:08:43.809583 32708 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:08:43.809630 32708 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:43.809667 32708 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:08:43.809682 32708 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:43.818078 32708 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.131:39879
I20260808 06:08:43.818114 446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.131:39879 every 8 connection(s)
I20260808 06:08:43.818941 32708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
I20260808 06:08:43.827430 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 32708
I20260808 06:08:43.827556 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance
I20260808 06:08:43.827881 1630 external_mini_cluster.cc:1428] Setting key 3c4ffe556da3487d5f85a6c8f5342bc2
Aug 08 06:08:43 dist-test-slave-zj8q krb5kdc[32108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169323, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.131@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:08:43.832218 32224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:43.820628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:54615 (local address 127.1.151.190:46459)
0808 06:08:43.820807 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:43.820812 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:43.821672 (+ 860us) server_negotiation.cc:407] Connection header received
0808 06:08:43.822995 (+ 1323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:43.822999 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:43.823047 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:43.823120 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:43.824264 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.824820 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.825974 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.826237 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.826324 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:43.828915 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:43.828918 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:43.828920 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:08:43.828946 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:43.830265 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:43.830755 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:43.830759 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:43.830761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:43.830843 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:43.831158 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:43.831162 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:43.831164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:43.831351 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:43.831437 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:43.831929 (+ 492us) server_negotiation.cc:299] Negotiation successful
0808 06:08:43.832058 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 06:08:43.832970 452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:43.821001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:46459 (local address 127.1.151.131:54615)
0808 06:08:43.821495 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:43.821544 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:43.822797 (+ 1253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:43.823275 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:43.823282 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:43.823635 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:08:43.824116 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:43.824131 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.824944 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.824947 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:43.825863 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:43.825873 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.826400 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.826404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:43.826552 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:43.827094 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:08:43.827118 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:08:43.828756 (+ 1638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:08:43.830383 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:08:43.830389 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:08:43.830405 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:08:43.830660 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:08:43.830925 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:08:43.830928 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:08:43.830930 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:08:43.831070 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:08:43.831451 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:08:43.831458 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:08:43.831696 (+ 238us) client_negotiation.cc:769] Sending connection context
0808 06:08:43.831864 (+ 168us) client_negotiation.cc:240] Negotiation successful
0808 06:08:43.832191 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":137,"threads_started":1}
I20260808 06:08:43.834075 447 heartbeater.cc:347] Connected to a master server at 127.1.151.190:46459
I20260808 06:08:43.834326 447 heartbeater.cc:464] Registering TS with master...
I20260808 06:08:43.834800 447 heartbeater.cc:511] Master 127.1.151.190:46459 requested a full tablet report, sending...
I20260808 06:08:43.835825 32172 ts_manager.cc:194] Registered new tserver with Master: 2ab96b0c917347febf972bc30d1f5165 (127.1.151.131:39879)
I20260808 06:08:43.836392 32172 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.151.131@KRBTEST.COM'} at 127.1.151.131:54615
I20260808 06:08:43.841548 1630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 06:08:43.850466 32224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:43.843461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35742 (local address 127.1.151.190:46459)
0808 06:08:43.843620 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:43.843624 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:43.843827 (+ 203us) server_negotiation.cc:407] Connection header received
0808 06:08:43.843954 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:43.843956 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:43.843994 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:43.844063 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:43.844837 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.845348 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.846213 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.846424 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.846511 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:43.847825 (+ 1314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:43.847828 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:43.847830 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:08:43.847851 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:43.848900 (+ 1049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:43.849297 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:43.849301 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:43.849303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:43.849377 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:43.849618 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:43.849621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:43.849623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:43.849833 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:43.849954 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:43.850210 (+ 256us) server_negotiation.cc:299] Negotiation successful
0808 06:08:43.850322 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 06:08:43.853076 32172 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35742:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 06:08:43.856108 32172 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 06:08:43.870133 465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:43.864208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:39879 (local address 127.0.0.1:42120)
0808 06:08:43.864651 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:43.864695 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:43.864815 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:43.866052 (+ 1237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:43.866058 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:43.866081 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:08:43.866237 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:43.866246 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.867336 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.867339 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:43.868522 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:43.868534 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.869600 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.869604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:43.869712 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:43.869753 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 06:08:43.869865 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 06:08:43.869985 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":65,"threads_started":1}
I20260808 06:08:43.870177 467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:43.864337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42120 (local address 127.1.151.131:39879)
0808 06:08:43.865500 (+ 1163us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:43.865515 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:43.865603 (+ 88us) server_negotiation.cc:407] Connection header received
0808 06:08:43.865685 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:43.865689 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:43.865880 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:43.866035 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:08:43.866352 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.867209 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.868661 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.869481 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.869589 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:43.869729 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:43.869905 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 06:08:43.870000 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1061,"thread_start_us":145,"threads_started":1}
I20260808 06:08:43.870774 468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:43.864870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:34705 (local address 127.0.0.1:50502)
0808 06:08:43.865215 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:43.865225 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:43.865297 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:43.866236 (+ 939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:43.866238 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:43.866256 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:08:43.866347 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:43.866353 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.867710 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.867714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:43.869113 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:43.869125 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.870395 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.870398 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:43.870464 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:43.870483 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 06:08:43.870564 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 06:08:43.870631 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":220,"thread_start_us":94,"threads_started":1}
I20260808 06:08:43.870854 469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:43.864872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45132 (local address 127.1.151.129:37231)
0808 06:08:43.865339 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:43.865352 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:43.865379 (+ 27us) server_negotiation.cc:407] Connection header received
0808 06:08:43.865657 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:43.865664 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:43.865886 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:43.866045 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:08:43.866695 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.867748 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.869204 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.869849 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.869938 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:43.870063 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:43.870618 (+ 555us) server_negotiation.cc:299] Negotiation successful
0808 06:08:43.870717 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":114,"threads_started":1}
I20260808 06:08:43.870512 466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:43.864352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:37231 (local address 127.0.0.1:45132)
0808 06:08:43.865159 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:43.865171 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:43.865459 (+ 288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:43.866473 (+ 1014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:43.866475 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:43.866492 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:08:43.866567 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:43.866572 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.867863 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.867866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:43.869084 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:43.869094 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.870124 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.870128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:43.870237 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:43.870263 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 06:08:43.870333 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 06:08:43.870390 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":747,"mutex_wait_us":55,"thread_start_us":96,"threads_started":1}
I20260808 06:08:43.872190 471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:43.865020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50502 (local address 127.1.151.130:34705)
0808 06:08:43.865591 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:43.865602 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:43.865621 (+ 19us) server_negotiation.cc:407] Connection header received
0808 06:08:43.865684 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:43.865689 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:43.865880 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:43.866040 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:08:43.866475 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.867572 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.869253 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.870058 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.870184 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:43.870329 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:43.870599 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 06:08:43.870694 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":102,"threads_started":1}
I20260808 06:08:43.873515 340 tablet_service.cc:1467] Processing CreateTablet for tablet 07c9c82e14344dc5b40bf8aed5322434 (DEFAULT_TABLE table=test-table [id=044014be35f84b88bef4ea45ceeaa143]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:08:43.873824 32363 tablet_service.cc:1467] Processing CreateTablet for tablet 07c9c82e14344dc5b40bf8aed5322434 (DEFAULT_TABLE table=test-table [id=044014be35f84b88bef4ea45ceeaa143]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:08:43.874589 340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07c9c82e14344dc5b40bf8aed5322434. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:08:43.874866 32363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07c9c82e14344dc5b40bf8aed5322434. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:08:43.874679 32591 tablet_service.cc:1467] Processing CreateTablet for tablet 07c9c82e14344dc5b40bf8aed5322434 (DEFAULT_TABLE table=test-table [id=044014be35f84b88bef4ea45ceeaa143]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:08:43.875571 32591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07c9c82e14344dc5b40bf8aed5322434. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:08:43.880718 475 tablet_bootstrap.cc:492] T 07c9c82e14344dc5b40bf8aed5322434 P 2ab96b0c917347febf972bc30d1f5165: Bootstrap starting.
I20260808 06:08:43.882294 477 tablet_bootstrap.cc:492] T 07c9c82e14344dc5b40bf8aed5322434 P 586e858ffc3f4c7baf9c27714e48d76b: Bootstrap starting.
I20260808 06:08:43.882436 480 tablet_bootstrap.cc:492] T 07c9c82e14344dc5b40bf8aed5322434 P 892f14f5be9c4191b000fe1151d8e54d: Bootstrap starting.
I20260808 06:08:43.883299 475 tablet_bootstrap.cc:654] T 07c9c82e14344dc5b40bf8aed5322434 P 2ab96b0c917347febf972bc30d1f5165: Neither blocks nor log segments found. Creating new log.
I20260808 06:08:43.884250 477 tablet_bootstrap.cc:654] T 07c9c82e14344dc5b40bf8aed5322434 P 586e858ffc3f4c7baf9c27714e48d76b: Neither blocks nor log segments found. Creating new log.
I20260808 06:08:43.884560 480 tablet_bootstrap.cc:654] T 07c9c82e14344dc5b40bf8aed5322434 P 892f14f5be9c4191b000fe1151d8e54d: Neither blocks nor log segments found. Creating new log.
I20260808 06:08:43.884639 475 log.cc:824] T 07c9c82e14344dc5b40bf8aed5322434 P 2ab96b0c917347febf972bc30d1f5165: Log is configured to *not* fsync() on all Append() calls
I20260808 06:08:43.885298 477 log.cc:824] T 07c9c82e14344dc5b40bf8aed5322434 P 586e858ffc3f4c7baf9c27714e48d76b: Log is configured to *not* fsync() on all Append() calls
I20260808 06:08:43.885648 480 log.cc:824] T 07c9c82e14344dc5b40bf8aed5322434 P 892f14f5be9c4191b000fe1151d8e54d: Log is configured to *not* fsync() on all Append() calls
I20260808 06:08:43.886655 475 tablet_bootstrap.cc:492] T 07c9c82e14344dc5b40bf8aed5322434 P 2ab96b0c917347febf972bc30d1f5165: No bootstrap required, opened a new log
I20260808 06:08:43.886849 475 ts_tablet_manager.cc:1397] T 07c9c82e14344dc5b40bf8aed5322434 P 2ab96b0c917347febf972bc30d1f5165: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 06:08:43.886977 477 tablet_bootstrap.cc:492] T 07c9c82e14344dc5b40bf8aed5322434 P 586e858ffc3f4c7baf9c27714e48d76b: No bootstrap required, opened a new log
I20260808 06:08:43.887153 477 ts_tablet_manager.cc:1397] T 07c9c82e14344dc5b40bf8aed5322434 P 586e858ffc3f4c7baf9c27714e48d76b: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 06:08:43.889886 475 raft_consensus.cc:348] T 07c9c82e14344dc5b40bf8aed5322434 P 2ab96b0c917347febf972bc30d1f5165 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab96b0c917347febf972bc30d1f5165" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 39879 } } peers { permanent_uuid: "586e858ffc3f4c7baf9c27714e48d76b" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 37231 } } peers { permanent_uuid: "892f14f5be9c4191b000fe1151d8e54d" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 34705 } }
I20260808 06:08:43.890110 475 raft_consensus.cc:374] T 07c9c82e14344dc5b40bf8aed5322434 P 2ab96b0c917347febf972bc30d1f5165 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:08:43.890146 475 raft_consensus.cc:729] T 07c9c82e14344dc5b40bf8aed5322434 P 2ab96b0c917347febf972bc30d1f5165 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab96b0c917347febf972bc30d1f5165, State: Initialized, Role: FOLLOWER
I20260808 06:08:43.890230 477 raft_consensus.cc:348] T 07c9c82e14344dc5b40bf8aed5322434 P 586e858ffc3f4c7baf9c27714e48d76b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab96b0c917347febf972bc30d1f5165" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 39879 } } peers { permanent_uuid: "586e858ffc3f4c7baf9c27714e48d76b" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 37231 } } peers { permanent_uuid: "892f14f5be9c4191b000fe1151d8e54d" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 34705 } }
I20260808 06:08:43.890429 477 raft_consensus.cc:374] T 07c9c82e14344dc5b40bf8aed5322434 P 586e858ffc3f4c7baf9c27714e48d76b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:08:43.890482 477 raft_consensus.cc:729] T 07c9c82e14344dc5b40bf8aed5322434 P 586e858ffc3f4c7baf9c27714e48d76b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 586e858ffc3f4c7baf9c27714e48d76b, State: Initialized, Role: FOLLOWER
I20260808 06:08:43.890594 475 consensus_queue.cc:260] T 07c9c82e14344dc5b40bf8aed5322434 P 2ab96b0c917347febf972bc30d1f5165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab96b0c917347febf972bc30d1f5165" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 39879 } } peers { permanent_uuid: "586e858ffc3f4c7baf9c27714e48d76b" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 37231 } } peers { permanent_uuid: "892f14f5be9c4191b000fe1151d8e54d" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 34705 } }
I20260808 06:08:43.890872 477 consensus_queue.cc:260] T 07c9c82e14344dc5b40bf8aed5322434 P 586e858ffc3f4c7baf9c27714e48d76b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab96b0c917347febf972bc30d1f5165" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 39879 } } peers { permanent_uuid: "586e858ffc3f4c7baf9c27714e48d76b" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 37231 } } peers { permanent_uuid: "892f14f5be9c4191b000fe1151d8e54d" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 34705 } }
I20260808 06:08:43.891242 447 heartbeater.cc:503] Master 127.1.151.190:46459 was elected leader, sending a full tablet report...
I20260808 06:08:43.891475 475 ts_tablet_manager.cc:1428] T 07c9c82e14344dc5b40bf8aed5322434 P 2ab96b0c917347febf972bc30d1f5165: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20260808 06:08:43.891762 477 ts_tablet_manager.cc:1428] T 07c9c82e14344dc5b40bf8aed5322434 P 586e858ffc3f4c7baf9c27714e48d76b: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 06:08:43.892269 32482 heartbeater.cc:503] Master 127.1.151.190:46459 was elected leader, sending a full tablet report...
I20260808 06:08:43.892915 480 tablet_bootstrap.cc:492] T 07c9c82e14344dc5b40bf8aed5322434 P 892f14f5be9c4191b000fe1151d8e54d: No bootstrap required, opened a new log
I20260808 06:08:43.893093 480 ts_tablet_manager.cc:1397] T 07c9c82e14344dc5b40bf8aed5322434 P 892f14f5be9c4191b000fe1151d8e54d: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s
I20260808 06:08:43.896484 480 raft_consensus.cc:348] T 07c9c82e14344dc5b40bf8aed5322434 P 892f14f5be9c4191b000fe1151d8e54d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab96b0c917347febf972bc30d1f5165" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 39879 } } peers { permanent_uuid: "586e858ffc3f4c7baf9c27714e48d76b" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 37231 } } peers { permanent_uuid: "892f14f5be9c4191b000fe1151d8e54d" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 34705 } }
I20260808 06:08:43.896693 480 raft_consensus.cc:374] T 07c9c82e14344dc5b40bf8aed5322434 P 892f14f5be9c4191b000fe1151d8e54d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:08:43.896742 480 raft_consensus.cc:729] T 07c9c82e14344dc5b40bf8aed5322434 P 892f14f5be9c4191b000fe1151d8e54d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 892f14f5be9c4191b000fe1151d8e54d, State: Initialized, Role: FOLLOWER
I20260808 06:08:43.897082 480 consensus_queue.cc:260] T 07c9c82e14344dc5b40bf8aed5322434 P 892f14f5be9c4191b000fe1151d8e54d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab96b0c917347febf972bc30d1f5165" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 39879 } } peers { permanent_uuid: "586e858ffc3f4c7baf9c27714e48d76b" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 37231 } } peers { permanent_uuid: "892f14f5be9c4191b000fe1151d8e54d" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 34705 } }
I20260808 06:08:43.897645 32702 heartbeater.cc:503] Master 127.1.151.190:46459 was elected leader, sending a full tablet report...
I20260808 06:08:43.897910 480 ts_tablet_manager.cc:1428] T 07c9c82e14344dc5b40bf8aed5322434 P 892f14f5be9c4191b000fe1151d8e54d: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 06:08:43.900941 485 raft_consensus.cc:482] T 07c9c82e14344dc5b40bf8aed5322434 P 2ab96b0c917347febf972bc30d1f5165 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:08:43.901065 485 raft_consensus.cc:504] T 07c9c82e14344dc5b40bf8aed5322434 P 2ab96b0c917347febf972bc30d1f5165 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab96b0c917347febf972bc30d1f5165" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 39879 } } peers { permanent_uuid: "586e858ffc3f4c7baf9c27714e48d76b" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 37231 } } peers { permanent_uuid: "892f14f5be9c4191b000fe1151d8e54d" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 34705 } }
I20260808 06:08:43.901959 485 leader_election.cc:288] T 07c9c82e14344dc5b40bf8aed5322434 P 2ab96b0c917347febf972bc30d1f5165 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 586e858ffc3f4c7baf9c27714e48d76b (127.1.151.129:37231), 892f14f5be9c4191b000fe1151d8e54d (127.1.151.130:34705)
W20260808 06:08:43.902021 32483 tablet.cc:2369] T 07c9c82e14344dc5b40bf8aed5322434 P 586e858ffc3f4c7baf9c27714e48d76b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 06:08:43.906302 471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:43.902297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:43979 (local address 127.1.151.130:34705)
0808 06:08:43.902653 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:43.902656 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:43.902884 (+ 228us) server_negotiation.cc:407] Connection header received
0808 06:08:43.902987 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:43.902990 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:43.903032 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:43.903097 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:08:43.903320 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.904023 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.905117 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.905720 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.905814 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:43.905855 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:43.906080 (+ 225us) server_negotiation.cc:299] Negotiation successful
0808 06:08:43.906186 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286}
I20260808 06:08:43.906302 469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:43.902406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:60337 (local address 127.1.151.129:37231)
0808 06:08:43.902693 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:43.902697 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:43.902709 (+ 12us) server_negotiation.cc:407] Connection header received
0808 06:08:43.902745 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:43.902747 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:43.902782 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:43.902857 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:08:43.903097 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.903875 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.905041 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.905650 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.905769 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:43.905822 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:43.906085 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 06:08:43.906189 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216}
I20260808 06:08:43.906307 452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:43.902172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:37231 (local address 127.1.151.131:60337)
0808 06:08:43.902414 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:43.902429 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:43.902507 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:43.902888 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:43.902890 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:43.902904 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:08:43.903016 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:43.903021 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.904002 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.904005 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:43.904925 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:43.904935 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.905765 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.905769 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:43.905879 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:43.905975 (+ 96us) client_negotiation.cc:769] Sending connection context
0808 06:08:43.906047 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 06:08:43.906171 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":102}
I20260808 06:08:43.906322 488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:43.902416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:34705 (local address 127.1.151.131:43979)
0808 06:08:43.902807 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:43.902817 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:43.902900 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:43.903117 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:43.903120 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:43.903142 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:08:43.903233 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:43.903238 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.904158 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.904162 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:43.905016 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:43.905027 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.905815 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.905818 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:43.905917 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:43.905979 (+ 62us) client_negotiation.cc:769] Sending connection context
0808 06:08:43.906047 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 06:08:43.906142 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":43,"thread_start_us":91,"threads_started":1}
I20260808 06:08:43.906955 32629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07c9c82e14344dc5b40bf8aed5322434" candidate_uuid: "2ab96b0c917347febf972bc30d1f5165" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "892f14f5be9c4191b000fe1151d8e54d" is_pre_election: true
I20260808 06:08:43.906956 32399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07c9c82e14344dc5b40bf8aed5322434" candidate_uuid: "2ab96b0c917347febf972bc30d1f5165" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "586e858ffc3f4c7baf9c27714e48d76b" is_pre_election: true
I20260808 06:08:43.907176 32629 raft_consensus.cc:2454] T 07c9c82e14344dc5b40bf8aed5322434 P 892f14f5be9c4191b000fe1151d8e54d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ab96b0c917347febf972bc30d1f5165 in term 0.
I20260808 06:08:43.907223 32399 raft_consensus.cc:2454] T 07c9c82e14344dc5b40bf8aed5322434 P 586e858ffc3f4c7baf9c27714e48d76b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ab96b0c917347febf972bc30d1f5165 in term 0.
I20260808 06:08:43.907580 32737 leader_election.cc:302] T 07c9c82e14344dc5b40bf8aed5322434 P 2ab96b0c917347febf972bc30d1f5165 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab96b0c917347febf972bc30d1f5165, 892f14f5be9c4191b000fe1151d8e54d; no voters:
I20260808 06:08:43.907826 485 raft_consensus.cc:2781] T 07c9c82e14344dc5b40bf8aed5322434 P 2ab96b0c917347febf972bc30d1f5165 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 06:08:43.907914 485 raft_consensus.cc:482] T 07c9c82e14344dc5b40bf8aed5322434 P 2ab96b0c917347febf972bc30d1f5165 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 06:08:43.907965 485 raft_consensus.cc:3037] T 07c9c82e14344dc5b40bf8aed5322434 P 2ab96b0c917347febf972bc30d1f5165 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:08:43.908891 485 raft_consensus.cc:504] T 07c9c82e14344dc5b40bf8aed5322434 P 2ab96b0c917347febf972bc30d1f5165 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab96b0c917347febf972bc30d1f5165" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 39879 } } peers { permanent_uuid: "586e858ffc3f4c7baf9c27714e48d76b" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 37231 } } peers { permanent_uuid: "892f14f5be9c4191b000fe1151d8e54d" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 34705 } }
I20260808 06:08:43.909181 485 leader_election.cc:288] T 07c9c82e14344dc5b40bf8aed5322434 P 2ab96b0c917347febf972bc30d1f5165 [CANDIDATE]: Term 1 election: Requested vote from peers 586e858ffc3f4c7baf9c27714e48d76b (127.1.151.129:37231), 892f14f5be9c4191b000fe1151d8e54d (127.1.151.130:34705)
I20260808 06:08:43.909512 32629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07c9c82e14344dc5b40bf8aed5322434" candidate_uuid: "2ab96b0c917347febf972bc30d1f5165" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "892f14f5be9c4191b000fe1151d8e54d"
I20260808 06:08:43.909512 32399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07c9c82e14344dc5b40bf8aed5322434" candidate_uuid: "2ab96b0c917347febf972bc30d1f5165" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "586e858ffc3f4c7baf9c27714e48d76b"
I20260808 06:08:43.909632 32399 raft_consensus.cc:3037] T 07c9c82e14344dc5b40bf8aed5322434 P 586e858ffc3f4c7baf9c27714e48d76b [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:08:43.909632 32629 raft_consensus.cc:3037] T 07c9c82e14344dc5b40bf8aed5322434 P 892f14f5be9c4191b000fe1151d8e54d [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:08:43.910856 32399 raft_consensus.cc:2454] T 07c9c82e14344dc5b40bf8aed5322434 P 586e858ffc3f4c7baf9c27714e48d76b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ab96b0c917347febf972bc30d1f5165 in term 1.
I20260808 06:08:43.910856 32629 raft_consensus.cc:2454] T 07c9c82e14344dc5b40bf8aed5322434 P 892f14f5be9c4191b000fe1151d8e54d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ab96b0c917347febf972bc30d1f5165 in term 1.
I20260808 06:08:43.911151 32739 leader_election.cc:302] T 07c9c82e14344dc5b40bf8aed5322434 P 2ab96b0c917347febf972bc30d1f5165 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab96b0c917347febf972bc30d1f5165, 586e858ffc3f4c7baf9c27714e48d76b; no voters:
I20260808 06:08:43.911367 485 raft_consensus.cc:2781] T 07c9c82e14344dc5b40bf8aed5322434 P 2ab96b0c917347febf972bc30d1f5165 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:08:43.911531 485 raft_consensus.cc:686] T 07c9c82e14344dc5b40bf8aed5322434 P 2ab96b0c917347febf972bc30d1f5165 [term 1 LEADER]: Becoming Leader. State: Replica: 2ab96b0c917347febf972bc30d1f5165, State: Running, Role: LEADER
I20260808 06:08:43.911796 485 consensus_queue.cc:237] T 07c9c82e14344dc5b40bf8aed5322434 P 2ab96b0c917347febf972bc30d1f5165 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab96b0c917347febf972bc30d1f5165" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 39879 } } peers { permanent_uuid: "586e858ffc3f4c7baf9c27714e48d76b" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 37231 } } peers { permanent_uuid: "892f14f5be9c4191b000fe1151d8e54d" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 34705 } }
I20260808 06:08:43.914810 32170 catalog_manager.cc:5495] T 07c9c82e14344dc5b40bf8aed5322434 P 2ab96b0c917347febf972bc30d1f5165 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ab96b0c917347febf972bc30d1f5165 (127.1.151.131). New cstate: current_term: 1 leader_uuid: "2ab96b0c917347febf972bc30d1f5165" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab96b0c917347febf972bc30d1f5165" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 39879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "586e858ffc3f4c7baf9c27714e48d76b" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 37231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "892f14f5be9c4191b000fe1151d8e54d" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 34705 } health_report { overall_health: UNKNOWN } } }
I20260808 06:08:43.934374 467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:43.930993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42128 (local address 127.1.151.131:39879)
0808 06:08:43.931102 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:43.931105 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:43.931118 (+ 13us) server_negotiation.cc:407] Connection header received
0808 06:08:43.931252 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:43.931255 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:43.931302 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:43.931381 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:08:43.931554 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.932037 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.932822 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:43.933071 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:43.933154 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:43.933481 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:08:43.933966 (+ 485us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:08:43.934054 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:43.934146 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 06:08:43.934225 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 06:08:43.939882 32629 raft_consensus.cc:1261] T 07c9c82e14344dc5b40bf8aed5322434 P 892f14f5be9c4191b000fe1151d8e54d [term 1 FOLLOWER]: Refusing update from remote peer 2ab96b0c917347febf972bc30d1f5165: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:08:43.939893 32399 raft_consensus.cc:1261] T 07c9c82e14344dc5b40bf8aed5322434 P 586e858ffc3f4c7baf9c27714e48d76b [term 1 FOLLOWER]: Refusing update from remote peer 2ab96b0c917347febf972bc30d1f5165: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:08:43.940564 490 consensus_queue.cc:1035] T 07c9c82e14344dc5b40bf8aed5322434 P 2ab96b0c917347febf972bc30d1f5165 [LEADER]: Connected to new peer: Peer: permanent_uuid: "892f14f5be9c4191b000fe1151d8e54d" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 34705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:08:43.940735 485 consensus_queue.cc:1035] T 07c9c82e14344dc5b40bf8aed5322434 P 2ab96b0c917347febf972bc30d1f5165 [LEADER]: Connected to new peer: Peer: permanent_uuid: "586e858ffc3f4c7baf9c27714e48d76b" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 37231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:08:43.949369 498 mvcc.cc:205] Tried to move back new op lower bound from 7316149550851112960 to 7316149550744961024. Current Snapshot: MvccSnapshot[applied={T|T < 7316149550851112960}]
I20260808 06:08:43.949656 496 mvcc.cc:205] Tried to move back new op lower bound from 7316149550851112960 to 7316149550744961024. Current Snapshot: MvccSnapshot[applied={T|T < 7316149550851112960}]
I20260808 06:08:43.956808 32172 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:08:43.956995 32172 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:08:43.959378 32172 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 28d25033676445518e2eaea1920f88f6: Sending DeleteTablet for 3 replicas of tablet 07c9c82e14344dc5b40bf8aed5322434
I20260808 06:08:43.960153 32363 tablet_service.cc:1514] Processing DeleteTablet for tablet 07c9c82e14344dc5b40bf8aed5322434 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:08:43 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:45132
I20260808 06:08:43.960284 340 tablet_service.cc:1514] Processing DeleteTablet for tablet 07c9c82e14344dc5b40bf8aed5322434 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:08:43 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:42120
I20260808 06:08:43.960384 32591 tablet_service.cc:1514] Processing DeleteTablet for tablet 07c9c82e14344dc5b40bf8aed5322434 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:08:43 UTC) from {username='kudu', principal='kudu/127.1.151.190@KRBTEST.COM'} at 127.0.0.1:50502
I20260808 06:08:43.960772 514 tablet_replica.cc:330] T 07c9c82e14344dc5b40bf8aed5322434 P 892f14f5be9c4191b000fe1151d8e54d: stopping tablet replica
I20260808 06:08:43.960932 512 tablet_replica.cc:330] T 07c9c82e14344dc5b40bf8aed5322434 P 2ab96b0c917347febf972bc30d1f5165: stopping tablet replica
I20260808 06:08:43.961113 514 raft_consensus.cc:2229] T 07c9c82e14344dc5b40bf8aed5322434 P 892f14f5be9c4191b000fe1151d8e54d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:08:43.961355 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 32260
I20260808 06:08:43.961257 512 raft_consensus.cc:2229] T 07c9c82e14344dc5b40bf8aed5322434 P 2ab96b0c917347febf972bc30d1f5165 [term 1 LEADER]: Raft consensus shutting down.
I20260808 06:08:43.962321 512 raft_consensus.cc:2258] T 07c9c82e14344dc5b40bf8aed5322434 P 2ab96b0c917347febf972bc30d1f5165 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:08:43.964123 514 raft_consensus.cc:2258] T 07c9c82e14344dc5b40bf8aed5322434 P 892f14f5be9c4191b000fe1151d8e54d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:08:43.965752 514 ts_tablet_manager.cc:1905] T 07c9c82e14344dc5b40bf8aed5322434 P 892f14f5be9c4191b000fe1151d8e54d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:08:43.966423 512 ts_tablet_manager.cc:1905] T 07c9c82e14344dc5b40bf8aed5322434 P 2ab96b0c917347febf972bc30d1f5165: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:08:43.968967 514 ts_tablet_manager.cc:1918] T 07c9c82e14344dc5b40bf8aed5322434 P 892f14f5be9c4191b000fe1151d8e54d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:08:43.969058 512 ts_tablet_manager.cc:1918] T 07c9c82e14344dc5b40bf8aed5322434 P 2ab96b0c917347febf972bc30d1f5165: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:08:43.969152 514 log.cc:1196] T 07c9c82e14344dc5b40bf8aed5322434 P 892f14f5be9c4191b000fe1151d8e54d: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-1/wal/wals/07c9c82e14344dc5b40bf8aed5322434
I20260808 06:08:43.969218 512 log.cc:1196] T 07c9c82e14344dc5b40bf8aed5322434 P 2ab96b0c917347febf972bc30d1f5165: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786169070948234-1630-0/minicluster-data/ts-2/wal/wals/07c9c82e14344dc5b40bf8aed5322434
I20260808 06:08:43.969623 512 ts_tablet_manager.cc:1939] T 07c9c82e14344dc5b40bf8aed5322434 P 2ab96b0c917347febf972bc30d1f5165: Deleting consensus metadata
I20260808 06:08:43.969641 514 ts_tablet_manager.cc:1939] T 07c9c82e14344dc5b40bf8aed5322434 P 892f14f5be9c4191b000fe1151d8e54d: Deleting consensus metadata
I20260808 06:08:43.970903 32149 catalog_manager.cc:4841] TS 2ab96b0c917347febf972bc30d1f5165 (127.1.151.131:39879): tablet 07c9c82e14344dc5b40bf8aed5322434 (table test-table [id=044014be35f84b88bef4ea45ceeaa143]) successfully deleted
I20260808 06:08:43.971133 32149 catalog_manager.cc:4841] TS 892f14f5be9c4191b000fe1151d8e54d (127.1.151.130:34705): tablet 07c9c82e14344dc5b40bf8aed5322434 (table test-table [id=044014be35f84b88bef4ea45ceeaa143]) successfully deleted
W20260808 06:08:43.976251 32151 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.151.129:37231 (error 108)
I20260808 06:08:43.977111 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 32490
I20260808 06:08:43.977327 466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:43.977006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:37231 (local address 127.0.0.1:45134)
0808 06:08:43.977154 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:43.977240 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.129:37231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 06:08:43.992130 32151 catalog_manager.cc:4584] TS 586e858ffc3f4c7baf9c27714e48d76b (127.1.151.129:37231): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 07c9c82e14344dc5b40bf8aed5322434: Network error: Client connection negotiation failed: client connection to 127.1.151.129:37231: connect: Connection refused (error 111)
I20260808 06:08:44.005291 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 32708
I20260808 06:08:44.016518 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 32130
2026-08-08T06:08:44Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2322 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T06:08:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:08:44Z Disabled control of system clock
I20260808 06:08:44.049381 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:43305
--webserver_interface=127.1.151.190
--webserver_port=0
--builtin_ntp_servers=127.1.151.148:32901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:43305
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 06:08:44.158624 524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:44.158896 524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:44.158931 524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:44.162843 524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:08:44.162899 524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:44.162912 524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:44.162926 524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:08:44.162940 524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:08:44.167776 524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:32901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:43305
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:43305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:44.168809 524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:44.170146 524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:08:44.178305 532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:44.178300 531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:44.178385 524 server_base.cc:1034] running on GCE node
W20260808 06:08:44.178344 534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:44.179086 524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:44.180143 524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:44.181303 524 hybrid_clock.cc:648] HybridClock initialized: now 1786169324181280 us; error 35 us; skew 500 ppm
I20260808 06:08:44.183174 524 webserver.cc:460] Webserver started at http://127.1.151.190:42029/ using document root <none> and password file <none>
I20260808 06:08:44.183728 524 fs_manager.cc:359] Metadata directory not provided
I20260808 06:08:44.183792 524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:08:44.183986 524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:08:44.186306 524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "88ff74125f85430dae0b676fc752c90b"
format_stamp: "Formatted at 2026-08-08 06:08:44 on dist-test-slave-zj8q"
server_key: "be4781c43cd5f7b312cb18ab6b2b0ada"
server_key_iv: "6f9faf0b63625ce3b9267fe2c09faf98"
server_key_version: "encryptionkey@0"
I20260808 06:08:44.186944 524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "88ff74125f85430dae0b676fc752c90b"
format_stamp: "Formatted at 2026-08-08 06:08:44 on dist-test-slave-zj8q"
server_key: "be4781c43cd5f7b312cb18ab6b2b0ada"
server_key_iv: "6f9faf0b63625ce3b9267fe2c09faf98"
server_key_version: "encryptionkey@0"
I20260808 06:08:44.190503 524 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 06:08:44.192888 544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:44.194043 524 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:08:44.194156 524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "88ff74125f85430dae0b676fc752c90b"
format_stamp: "Formatted at 2026-08-08 06:08:44 on dist-test-slave-zj8q"
server_key: "be4781c43cd5f7b312cb18ab6b2b0ada"
server_key_iv: "6f9faf0b63625ce3b9267fe2c09faf98"
server_key_version: "encryptionkey@0"
I20260808 06:08:44.194300 524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:08:44.208892 524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:08:44.209517 524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:08:44.209681 524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:08:44.216252 524 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:43305
I20260808 06:08:44.216282 629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:43305 every 8 connection(s)
I20260808 06:08:44.217119 524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:08:44.220399 630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:08:44.225188 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 524
I20260808 06:08:44.225318 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:08:44.225572 1630 external_mini_cluster.cc:1428] Setting key 946dabee16ffdd9938e13281410120f0
I20260808 06:08:44.225858 630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88ff74125f85430dae0b676fc752c90b: Bootstrap starting.
I20260808 06:08:44.228554 630 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88ff74125f85430dae0b676fc752c90b: Neither blocks nor log segments found. Creating new log.
I20260808 06:08:44.229415 630 log.cc:824] T 00000000000000000000000000000000 P 88ff74125f85430dae0b676fc752c90b: Log is configured to *not* fsync() on all Append() calls
I20260808 06:08:44.231559 630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88ff74125f85430dae0b676fc752c90b: No bootstrap required, opened a new log
I20260808 06:08:44.235560 634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:44.226975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58250 (local address 127.1.151.190:43305)
0808 06:08:44.227636 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:44.227662 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:44.227705 (+ 43us) server_negotiation.cc:407] Connection header received
0808 06:08:44.228563 (+ 858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:44.228588 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:44.229085 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:44.229439 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:44.229872 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.231231 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.232113 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.232779 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.232947 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:44.233734 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:44.233740 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:44.233756 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:08:44.234295 (+ 539us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:44.234368 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:08:44.234396 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:44.234485 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:44.234651 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 06:08:44.234867 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":112,"threads_started":1}
I20260808 06:08:44.235754 630 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88ff74125f85430dae0b676fc752c90b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88ff74125f85430dae0b676fc752c90b" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 43305 } }
I20260808 06:08:44.235987 630 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88ff74125f85430dae0b676fc752c90b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:08:44.236047 630 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88ff74125f85430dae0b676fc752c90b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ff74125f85430dae0b676fc752c90b, State: Initialized, Role: FOLLOWER
I20260808 06:08:44.236469 630 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88ff74125f85430dae0b676fc752c90b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88ff74125f85430dae0b676fc752c90b" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 43305 } }
I20260808 06:08:44.236605 630 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88ff74125f85430dae0b676fc752c90b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:08:44.236673 630 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88ff74125f85430dae0b676fc752c90b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:08:44.236773 630 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88ff74125f85430dae0b676fc752c90b [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:08:44.237663 630 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88ff74125f85430dae0b676fc752c90b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88ff74125f85430dae0b676fc752c90b" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 43305 } }
I20260808 06:08:44.237982 630 leader_election.cc:302] T 00000000000000000000000000000000 P 88ff74125f85430dae0b676fc752c90b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88ff74125f85430dae0b676fc752c90b; no voters:
I20260808 06:08:44.238278 630 leader_election.cc:288] T 00000000000000000000000000000000 P 88ff74125f85430dae0b676fc752c90b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:08:44.238354 638 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88ff74125f85430dae0b676fc752c90b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:08:44.238535 638 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88ff74125f85430dae0b676fc752c90b [term 1 LEADER]: Becoming Leader. State: Replica: 88ff74125f85430dae0b676fc752c90b, State: Running, Role: LEADER
I20260808 06:08:44.238852 638 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88ff74125f85430dae0b676fc752c90b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88ff74125f85430dae0b676fc752c90b" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 43305 } }
I20260808 06:08:44.239519 630 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88ff74125f85430dae0b676fc752c90b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:08:44.240477 640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88ff74125f85430dae0b676fc752c90b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88ff74125f85430dae0b676fc752c90b. Latest consensus state: current_term: 1 leader_uuid: "88ff74125f85430dae0b676fc752c90b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88ff74125f85430dae0b676fc752c90b" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 43305 } } }
I20260808 06:08:44.240623 640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88ff74125f85430dae0b676fc752c90b [sys.catalog]: This master's current role is: LEADER
I20260808 06:08:44.240993 649 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:08:44.241639 639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88ff74125f85430dae0b676fc752c90b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88ff74125f85430dae0b676fc752c90b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88ff74125f85430dae0b676fc752c90b" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 43305 } } }
I20260808 06:08:44.241727 639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88ff74125f85430dae0b676fc752c90b [sys.catalog]: This master's current role is: LEADER
I20260808 06:08:44.244031 649 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:08:44.249239 649 catalog_manager.cc:1347] Generated new cluster ID: 742800fe57144e8f8e01b5da41f2ae02
I20260808 06:08:44.249327 649 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:08:44.262538 649 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:08:44.263527 649 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:08:44.268054 649 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88ff74125f85430dae0b676fc752c90b: Generated new TSK 0
I20260808 06:08:44.268616 649 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 06:08:44.275004 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.129:0
--local_ip_for_outbound_sockets=127.1.151.129
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:43305
--builtin_ntp_servers=127.1.151.148:32901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 06:08:44.386795 663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:44.387022 663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:44.387056 663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:44.390892 663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:44.390937 663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:44.390987 663 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.129
I20260808 06:08:44.395545 663 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:32901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.151.129
--webserver_port=0
--tserver_master_addrs=127.1.151.190:43305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.129
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:44.396427 663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:44.397714 663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:08:44.406837 677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:44.406849 672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:44.406838 673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:44.407101 663 server_base.cc:1034] running on GCE node
I20260808 06:08:44.407482 663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:44.408054 663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:44.409215 663 hybrid_clock.cc:648] HybridClock initialized: now 1786169324409190 us; error 36 us; skew 500 ppm
I20260808 06:08:44.410941 663 webserver.cc:460] Webserver started at http://127.1.151.129:45531/ using document root <none> and password file <none>
I20260808 06:08:44.411484 663 fs_manager.cc:359] Metadata directory not provided
I20260808 06:08:44.411543 663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:08:44.411725 663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:08:44.413664 663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-0/data/instance:
uuid: "839d7002c8844d93b77682ee8e2204a4"
format_stamp: "Formatted at 2026-08-08 06:08:44 on dist-test-slave-zj8q"
server_key: "5ca124f93458821f95e671f55eeb24c8"
server_key_iv: "edf436cd8a59a06eada5c61e04a9a41e"
server_key_version: "encryptionkey@0"
I20260808 06:08:44.414227 663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance:
uuid: "839d7002c8844d93b77682ee8e2204a4"
format_stamp: "Formatted at 2026-08-08 06:08:44 on dist-test-slave-zj8q"
server_key: "5ca124f93458821f95e671f55eeb24c8"
server_key_iv: "edf436cd8a59a06eada5c61e04a9a41e"
server_key_version: "encryptionkey@0"
I20260808 06:08:44.417647 663 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 06:08:44.419909 689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:44.421115 663 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:08:44.421258 663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-0/wal
uuid: "839d7002c8844d93b77682ee8e2204a4"
format_stamp: "Formatted at 2026-08-08 06:08:44 on dist-test-slave-zj8q"
server_key: "5ca124f93458821f95e671f55eeb24c8"
server_key_iv: "edf436cd8a59a06eada5c61e04a9a41e"
server_key_version: "encryptionkey@0"
I20260808 06:08:44.421352 663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:08:44.433705 663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:08:44.434317 663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:08:44.434473 663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:08:44.434994 663 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:08:44.435860 663 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:08:44.435907 663 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:44.435956 663 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:08:44.435986 663 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:44.444494 663 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.129:43645
I20260808 06:08:44.444532 864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.129:43645 every 8 connection(s)
I20260808 06:08:44.445322 663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-0/data/info.pb
I20260808 06:08:44.452242 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 663
I20260808 06:08:44.452349 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-0/wal/instance
I20260808 06:08:44.452605 1630 external_mini_cluster.cc:1428] Setting key 768b0ed31e72a835bfcc5bdf74c10ee2
I20260808 06:08:44.454403 634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:44.447103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.129:57897 (local address 127.1.151.190:43305)
0808 06:08:44.447251 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:44.447254 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:44.448272 (+ 1018us) server_negotiation.cc:407] Connection header received
0808 06:08:44.449650 (+ 1378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:44.449653 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:44.449713 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:44.449789 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:44.450746 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.451297 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.452457 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.452700 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.452801 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:44.453911 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:44.453915 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:44.453918 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:08:44.453931 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:44.453953 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:08:44.453962 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:44.454019 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:44.454194 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 06:08:44.454258 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 06:08:44.454665 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.130:0
--local_ip_for_outbound_sockets=127.1.151.130
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:43305
--builtin_ntp_servers=127.1.151.148:32901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 06:08:44.455279 869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:44.447489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:43305 (local address 127.1.151.129:57897)
0808 06:08:44.448119 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:44.448166 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:44.449438 (+ 1272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:44.449965 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:44.449972 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:44.450053 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:08:44.450573 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:44.450586 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.451423 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.451426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:44.452330 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:44.452343 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.452857 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.452862 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:44.453077 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:44.453644 (+ 567us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 06:08:44.453647 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:08:44.453698 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 06:08:44.453776 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:08:44.454031 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:08:44.454042 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:08:44.454048 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 06:08:44.454179 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 06:08:44.454453 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":115,"threads_started":1}
I20260808 06:08:44.456401 865 heartbeater.cc:347] Connected to a master server at 127.1.151.190:43305
I20260808 06:08:44.456625 865 heartbeater.cc:464] Registering TS with master...
I20260808 06:08:44.457108 865 heartbeater.cc:511] Master 127.1.151.190:43305 requested a full tablet report, sending...
I20260808 06:08:44.458573 572 ts_manager.cc:194] Registered new tserver with Master: 839d7002c8844d93b77682ee8e2204a4 (127.1.151.129:43645)
I20260808 06:08:44.459700 572 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.151.129:57897
W20260808 06:08:44.569664 872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:44.569912 872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:44.569949 872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:44.573966 872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:44.574018 872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:44.574076 872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.130
I20260808 06:08:44.578766 872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:32901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.151.130
--webserver_port=0
--tserver_master_addrs=127.1.151.190:43305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.130
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:44.579736 872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:44.581103 872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:08:44.590523 880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:44.590652 881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:44.590533 883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:44.590691 872 server_base.cc:1034] running on GCE node
I20260808 06:08:44.591261 872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:44.591861 872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:44.593040 872 hybrid_clock.cc:648] HybridClock initialized: now 1786169324592996 us; error 57 us; skew 500 ppm
I20260808 06:08:44.603137 872 webserver.cc:460] Webserver started at http://127.1.151.130:41013/ using document root <none> and password file <none>
I20260808 06:08:44.603799 872 fs_manager.cc:359] Metadata directory not provided
I20260808 06:08:44.603868 872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:08:44.604036 872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:08:44.606041 872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-1/data/instance:
uuid: "c76cd0b6fb8b4790ad3264dff35ec16b"
format_stamp: "Formatted at 2026-08-08 06:08:44 on dist-test-slave-zj8q"
server_key: "85452f86dabb99de632a19db6fcb2018"
server_key_iv: "715e65274691c5775e08395b69e57f7e"
server_key_version: "encryptionkey@0"
I20260808 06:08:44.606622 872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance:
uuid: "c76cd0b6fb8b4790ad3264dff35ec16b"
format_stamp: "Formatted at 2026-08-08 06:08:44 on dist-test-slave-zj8q"
server_key: "85452f86dabb99de632a19db6fcb2018"
server_key_iv: "715e65274691c5775e08395b69e57f7e"
server_key_version: "encryptionkey@0"
I20260808 06:08:44.610014 872 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 06:08:44.612391 892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:44.613580 872 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:08:44.613684 872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-1/wal
uuid: "c76cd0b6fb8b4790ad3264dff35ec16b"
format_stamp: "Formatted at 2026-08-08 06:08:44 on dist-test-slave-zj8q"
server_key: "85452f86dabb99de632a19db6fcb2018"
server_key_iv: "715e65274691c5775e08395b69e57f7e"
server_key_version: "encryptionkey@0"
I20260808 06:08:44.613773 872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:08:44.634784 872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:08:44.635380 872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:08:44.635521 872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:08:44.636039 872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:08:44.636920 872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:08:44.636967 872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:44.637004 872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:08:44.637032 872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:44.645864 872 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.130:45555
I20260808 06:08:44.645903 1060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.130:45555 every 8 connection(s)
I20260808 06:08:44.646790 872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-1/data/info.pb
I20260808 06:08:44.651988 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 872
I20260808 06:08:44.652092 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-1/wal/instance
I20260808 06:08:44.652356 1630 external_mini_cluster.cc:1428] Setting key af6f05acf091b3f4490033f145e10a32
I20260808 06:08:44.654382 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.151.131:0
--local_ip_for_outbound_sockets=127.1.151.131
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:43305
--builtin_ntp_servers=127.1.151.148:32901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 06:08:44.655890 634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:44.648495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.130:59937 (local address 127.1.151.190:43305)
0808 06:08:44.648616 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:44.648619 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:44.649618 (+ 999us) server_negotiation.cc:407] Connection header received
0808 06:08:44.651037 (+ 1419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:44.651040 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:44.651091 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:44.651167 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:44.652040 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.652660 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.654009 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.654278 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.654370 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:44.655429 (+ 1059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:44.655433 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:44.655436 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:08:44.655450 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:44.655469 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:08:44.655477 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:44.655536 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:44.655683 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 06:08:44.655745 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 06:08:44.656683 1066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:44.648882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:43305 (local address 127.1.151.130:59937)
0808 06:08:44.649492 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:44.649531 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:44.650815 (+ 1284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:44.651309 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:44.651316 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:44.651393 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:08:44.651886 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:44.651899 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.652795 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.652798 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:44.653886 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:44.653897 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.654420 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.654425 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:44.654616 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:44.655177 (+ 561us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 06:08:44.655178 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:08:44.655216 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 06:08:44.655278 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:08:44.655546 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:08:44.655556 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:08:44.655562 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 06:08:44.655675 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 06:08:44.655936 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":105,"threads_started":1}
I20260808 06:08:44.657780 1062 heartbeater.cc:347] Connected to a master server at 127.1.151.190:43305
I20260808 06:08:44.657990 1062 heartbeater.cc:464] Registering TS with master...
I20260808 06:08:44.658499 1062 heartbeater.cc:511] Master 127.1.151.190:43305 requested a full tablet report, sending...
I20260808 06:08:44.659507 572 ts_manager.cc:194] Registered new tserver with Master: c76cd0b6fb8b4790ad3264dff35ec16b (127.1.151.130:45555)
I20260808 06:08:44.660048 572 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.151.130:59937
W20260808 06:08:44.764289 1068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:44.764529 1068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:44.764580 1068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:44.768494 1068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:44.768540 1068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:44.768613 1068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.151.131
I20260808 06:08:44.773262 1068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:32901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.151.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.151.131
--webserver_port=0
--tserver_master_addrs=127.1.151.190:43305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.151.131
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:44.774245 1068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:44.775532 1068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:08:44.784693 1078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:44.784654 1074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:44.784719 1082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:44.784938 1068 server_base.cc:1034] running on GCE node
I20260808 06:08:44.785311 1068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:44.785899 1068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:44.787050 1068 hybrid_clock.cc:648] HybridClock initialized: now 1786169324787021 us; error 39 us; skew 500 ppm
I20260808 06:08:44.788655 1068 webserver.cc:460] Webserver started at http://127.1.151.131:32889/ using document root <none> and password file <none>
I20260808 06:08:44.789197 1068 fs_manager.cc:359] Metadata directory not provided
I20260808 06:08:44.789259 1068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:08:44.789458 1068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:08:44.791424 1068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-2/data/instance:
uuid: "48180e07e38b4abebf8fe682971a1422"
format_stamp: "Formatted at 2026-08-08 06:08:44 on dist-test-slave-zj8q"
server_key: "1a88bf68749ddb93772184d1c0719e43"
server_key_iv: "89d428e30dbf5f1b6d81b90964e1f055"
server_key_version: "encryptionkey@0"
I20260808 06:08:44.791960 1068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance:
uuid: "48180e07e38b4abebf8fe682971a1422"
format_stamp: "Formatted at 2026-08-08 06:08:44 on dist-test-slave-zj8q"
server_key: "1a88bf68749ddb93772184d1c0719e43"
server_key_iv: "89d428e30dbf5f1b6d81b90964e1f055"
server_key_version: "encryptionkey@0"
I20260808 06:08:44.795331 1068 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 06:08:44.797534 1090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:44.798750 1068 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:08:44.798862 1068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-2/wal
uuid: "48180e07e38b4abebf8fe682971a1422"
format_stamp: "Formatted at 2026-08-08 06:08:44 on dist-test-slave-zj8q"
server_key: "1a88bf68749ddb93772184d1c0719e43"
server_key_iv: "89d428e30dbf5f1b6d81b90964e1f055"
server_key_version: "encryptionkey@0"
I20260808 06:08:44.798960 1068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:08:44.819382 1068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:08:44.819974 1068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:08:44.820134 1068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:08:44.820672 1068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 06:08:44.821583 1068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 06:08:44.821630 1068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:44.821673 1068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 06:08:44.821702 1068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:44.830166 1068 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.131:37129
I20260808 06:08:44.830207 1280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.131:37129 every 8 connection(s)
I20260808 06:08:44.830998 1068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-2/data/info.pb
I20260808 06:08:44.831391 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 1068
I20260808 06:08:44.831503 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-2/wal/instance
I20260808 06:08:44.831817 1630 external_mini_cluster.cc:1428] Setting key 30a295425eb7f1b95d0baefbea5bb469
I20260808 06:08:44.839905 634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:44.832761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:56691 (local address 127.1.151.190:43305)
0808 06:08:44.832886 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:44.832890 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:44.834114 (+ 1224us) server_negotiation.cc:407] Connection header received
0808 06:08:44.835512 (+ 1398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:44.835515 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:44.835560 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:44.835624 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:44.836472 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.837023 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.838131 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.838405 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.838483 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:44.839447 (+ 964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:44.839450 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:44.839452 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:08:44.839464 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:44.839484 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:08:44.839492 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:44.839548 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:44.839735 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 06:08:44.839789 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 06:08:44.840723 1283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:44.833259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:43305 (local address 127.1.151.131:56691)
0808 06:08:44.833967 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:44.834012 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:44.835335 (+ 1323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:44.835789 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:44.835795 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:44.835873 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:08:44.836339 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:44.836350 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.837119 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.837121 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:44.838020 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:44.838030 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.838531 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.838534 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:44.838685 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:44.839208 (+ 523us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 06:08:44.839210 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:08:44.839247 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 06:08:44.839309 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:08:44.839553 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:08:44.839558 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:08:44.839562 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 06:08:44.839709 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 06:08:44.839929 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":129,"threads_started":1}
I20260808 06:08:44.841804 1281 heartbeater.cc:347] Connected to a master server at 127.1.151.190:43305
I20260808 06:08:44.842022 1281 heartbeater.cc:464] Registering TS with master...
I20260808 06:08:44.842551 1281 heartbeater.cc:511] Master 127.1.151.190:43305 requested a full tablet report, sending...
I20260808 06:08:44.843526 572 ts_manager.cc:194] Registered new tserver with Master: 48180e07e38b4abebf8fe682971a1422 (127.1.151.131:37129)
I20260808 06:08:44.844069 572 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.151.131:56691
I20260808 06:08:44.845733 1630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 06:08:44.851208 634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:44.847658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58256 (local address 127.1.151.190:43305)
0808 06:08:44.847766 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:44.847771 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:44.847951 (+ 180us) server_negotiation.cc:407] Connection header received
0808 06:08:44.848062 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:44.848066 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:44.848123 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:44.848185 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:44.848446 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.848933 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.849760 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.849969 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.850052 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:44.850836 (+ 784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:44.850840 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:44.850843 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:08:44.850855 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:44.850874 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:08:44.850882 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:44.850937 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:44.851022 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 06:08:44.851073 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 06:08:44.853760 572 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58256:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 06:08:44.856616 572 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 06:08:44.870083 1295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:44.864332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:43645 (local address 127.0.0.1:52110)
0808 06:08:44.865139 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:44.865186 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:44.865321 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:44.865857 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:44.865862 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:44.865884 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:08:44.866021 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:44.866029 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.867217 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.867221 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:44.868618 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:44.868630 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.869558 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.869561 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:44.869667 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:44.869704 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 06:08:44.869813 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 06:08:44.869930 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"mutex_wait_us":40,"thread_start_us":77,"threads_started":1}
I20260808 06:08:44.870088 1296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:44.864629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52110 (local address 127.1.151.129:43645)
0808 06:08:44.864978 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:44.864990 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:44.865272 (+ 282us) server_negotiation.cc:407] Connection header received
0808 06:08:44.865471 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:44.865477 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:44.865708 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:44.865862 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:08:44.866172 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.867110 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.868767 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.869439 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.869545 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:44.869741 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:44.869844 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 06:08:44.869938 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":117,"threads_started":1}
I20260808 06:08:44.870298 1294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:44.865009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:45555 (local address 127.0.0.1:43194)
0808 06:08:44.865545 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:44.865556 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:44.865665 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:44.866317 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:44.866320 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:44.866339 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:08:44.866430 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:44.866436 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.867636 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.867639 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:44.868613 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:44.868624 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.869899 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.869903 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:44.869989 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:44.870011 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 06:08:44.870073 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 06:08:44.870143 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"mutex_wait_us":23,"thread_start_us":96,"threads_started":1}
I20260808 06:08:44.870734 1299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:44.865411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43194 (local address 127.1.151.130:45555)
0808 06:08:44.865862 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:44.865873 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:44.865894 (+ 21us) server_negotiation.cc:407] Connection header received
0808 06:08:44.865965 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:44.865971 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:44.866180 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:44.866319 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:08:44.866547 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.867513 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.868752 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.869547 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.869659 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:44.869831 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:44.870469 (+ 638us) server_negotiation.cc:299] Negotiation successful
0808 06:08:44.870569 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":137,"threads_started":1}
I20260808 06:08:44.870756 1298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:44.864957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46374 (local address 127.1.151.131:37129)
0808 06:08:44.865390 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:44.865402 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:44.865422 (+ 20us) server_negotiation.cc:407] Connection header received
0808 06:08:44.866204 (+ 782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:44.866210 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:44.866379 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:44.866491 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:08:44.866900 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.867781 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.869147 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.869882 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.869974 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:44.870144 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:44.870467 (+ 323us) server_negotiation.cc:299] Negotiation successful
0808 06:08:44.870618 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":155,"threads_started":1}
I20260808 06:08:44.871078 1297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:44.864451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.131:37129 (local address 127.0.0.1:46374)
0808 06:08:44.865329 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:44.865342 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:44.866026 (+ 684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:44.866675 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:44.866678 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:44.866695 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:08:44.866777 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:44.866783 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.867900 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.867903 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:44.869011 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:44.869020 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.870224 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.870227 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:44.870311 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:44.870334 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 06:08:44.870881 (+ 547us) client_negotiation.cc:240] Negotiation successful
0808 06:08:44.870960 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":740,"mutex_wait_us":23,"thread_start_us":104,"threads_started":1}
I20260808 06:08:44.873224 762 tablet_service.cc:1467] Processing CreateTablet for tablet c5fa0f3f2a6a4ebaa353868230609749 (DEFAULT_TABLE table=test-table [id=871dbeeb9cb14f7094b0652590b6da77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:08:44.873227 967 tablet_service.cc:1467] Processing CreateTablet for tablet c5fa0f3f2a6a4ebaa353868230609749 (DEFAULT_TABLE table=test-table [id=871dbeeb9cb14f7094b0652590b6da77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:08:44.874133 762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5fa0f3f2a6a4ebaa353868230609749. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:08:44.874197 967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5fa0f3f2a6a4ebaa353868230609749. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:08:44.875461 1169 tablet_service.cc:1467] Processing CreateTablet for tablet c5fa0f3f2a6a4ebaa353868230609749 (DEFAULT_TABLE table=test-table [id=871dbeeb9cb14f7094b0652590b6da77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 06:08:44.876375 1169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5fa0f3f2a6a4ebaa353868230609749. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:08:44.880808 1301 tablet_bootstrap.cc:492] T c5fa0f3f2a6a4ebaa353868230609749 P 839d7002c8844d93b77682ee8e2204a4: Bootstrap starting.
I20260808 06:08:44.881592 1302 tablet_bootstrap.cc:492] T c5fa0f3f2a6a4ebaa353868230609749 P c76cd0b6fb8b4790ad3264dff35ec16b: Bootstrap starting.
I20260808 06:08:44.882794 1301 tablet_bootstrap.cc:654] T c5fa0f3f2a6a4ebaa353868230609749 P 839d7002c8844d93b77682ee8e2204a4: Neither blocks nor log segments found. Creating new log.
I20260808 06:08:44.883299 1303 tablet_bootstrap.cc:492] T c5fa0f3f2a6a4ebaa353868230609749 P 48180e07e38b4abebf8fe682971a1422: Bootstrap starting.
I20260808 06:08:44.883642 1301 log.cc:824] T c5fa0f3f2a6a4ebaa353868230609749 P 839d7002c8844d93b77682ee8e2204a4: Log is configured to *not* fsync() on all Append() calls
I20260808 06:08:44.883711 1302 tablet_bootstrap.cc:654] T c5fa0f3f2a6a4ebaa353868230609749 P c76cd0b6fb8b4790ad3264dff35ec16b: Neither blocks nor log segments found. Creating new log.
I20260808 06:08:44.884491 1302 log.cc:824] T c5fa0f3f2a6a4ebaa353868230609749 P c76cd0b6fb8b4790ad3264dff35ec16b: Log is configured to *not* fsync() on all Append() calls
I20260808 06:08:44.885353 1303 tablet_bootstrap.cc:654] T c5fa0f3f2a6a4ebaa353868230609749 P 48180e07e38b4abebf8fe682971a1422: Neither blocks nor log segments found. Creating new log.
I20260808 06:08:44.885907 1301 tablet_bootstrap.cc:492] T c5fa0f3f2a6a4ebaa353868230609749 P 839d7002c8844d93b77682ee8e2204a4: No bootstrap required, opened a new log
I20260808 06:08:44.886091 1301 ts_tablet_manager.cc:1397] T c5fa0f3f2a6a4ebaa353868230609749 P 839d7002c8844d93b77682ee8e2204a4: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 06:08:44.886169 1303 log.cc:824] T c5fa0f3f2a6a4ebaa353868230609749 P 48180e07e38b4abebf8fe682971a1422: Log is configured to *not* fsync() on all Append() calls
I20260808 06:08:44.886312 1302 tablet_bootstrap.cc:492] T c5fa0f3f2a6a4ebaa353868230609749 P c76cd0b6fb8b4790ad3264dff35ec16b: No bootstrap required, opened a new log
I20260808 06:08:44.886492 1302 ts_tablet_manager.cc:1397] T c5fa0f3f2a6a4ebaa353868230609749 P c76cd0b6fb8b4790ad3264dff35ec16b: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260808 06:08:44.888092 1303 tablet_bootstrap.cc:492] T c5fa0f3f2a6a4ebaa353868230609749 P 48180e07e38b4abebf8fe682971a1422: No bootstrap required, opened a new log
I20260808 06:08:44.888283 1303 ts_tablet_manager.cc:1397] T c5fa0f3f2a6a4ebaa353868230609749 P 48180e07e38b4abebf8fe682971a1422: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 06:08:44.889701 1302 raft_consensus.cc:348] T c5fa0f3f2a6a4ebaa353868230609749 P c76cd0b6fb8b4790ad3264dff35ec16b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "839d7002c8844d93b77682ee8e2204a4" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 43645 } } peers { permanent_uuid: "48180e07e38b4abebf8fe682971a1422" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 37129 } } peers { permanent_uuid: "c76cd0b6fb8b4790ad3264dff35ec16b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45555 } }
I20260808 06:08:44.889906 1302 raft_consensus.cc:374] T c5fa0f3f2a6a4ebaa353868230609749 P c76cd0b6fb8b4790ad3264dff35ec16b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:08:44.889966 1302 raft_consensus.cc:729] T c5fa0f3f2a6a4ebaa353868230609749 P c76cd0b6fb8b4790ad3264dff35ec16b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c76cd0b6fb8b4790ad3264dff35ec16b, State: Initialized, Role: FOLLOWER
I20260808 06:08:44.890089 1301 raft_consensus.cc:348] T c5fa0f3f2a6a4ebaa353868230609749 P 839d7002c8844d93b77682ee8e2204a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "839d7002c8844d93b77682ee8e2204a4" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 43645 } } peers { permanent_uuid: "48180e07e38b4abebf8fe682971a1422" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 37129 } } peers { permanent_uuid: "c76cd0b6fb8b4790ad3264dff35ec16b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45555 } }
I20260808 06:08:44.890367 1301 raft_consensus.cc:374] T c5fa0f3f2a6a4ebaa353868230609749 P 839d7002c8844d93b77682ee8e2204a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:08:44.890441 1301 raft_consensus.cc:729] T c5fa0f3f2a6a4ebaa353868230609749 P 839d7002c8844d93b77682ee8e2204a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 839d7002c8844d93b77682ee8e2204a4, State: Initialized, Role: FOLLOWER
I20260808 06:08:44.890405 1302 consensus_queue.cc:260] T c5fa0f3f2a6a4ebaa353868230609749 P c76cd0b6fb8b4790ad3264dff35ec16b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "839d7002c8844d93b77682ee8e2204a4" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 43645 } } peers { permanent_uuid: "48180e07e38b4abebf8fe682971a1422" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 37129 } } peers { permanent_uuid: "c76cd0b6fb8b4790ad3264dff35ec16b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45555 } }
I20260808 06:08:44.890856 1301 consensus_queue.cc:260] T c5fa0f3f2a6a4ebaa353868230609749 P 839d7002c8844d93b77682ee8e2204a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "839d7002c8844d93b77682ee8e2204a4" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 43645 } } peers { permanent_uuid: "48180e07e38b4abebf8fe682971a1422" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 37129 } } peers { permanent_uuid: "c76cd0b6fb8b4790ad3264dff35ec16b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45555 } }
I20260808 06:08:44.891067 1062 heartbeater.cc:503] Master 127.1.151.190:43305 was elected leader, sending a full tablet report...
I20260808 06:08:44.891388 1302 ts_tablet_manager.cc:1428] T c5fa0f3f2a6a4ebaa353868230609749 P c76cd0b6fb8b4790ad3264dff35ec16b: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20260808 06:08:44.891741 1301 ts_tablet_manager.cc:1428] T c5fa0f3f2a6a4ebaa353868230609749 P 839d7002c8844d93b77682ee8e2204a4: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 06:08:44.892041 865 heartbeater.cc:503] Master 127.1.151.190:43305 was elected leader, sending a full tablet report...
I20260808 06:08:44.892158 1303 raft_consensus.cc:348] T c5fa0f3f2a6a4ebaa353868230609749 P 48180e07e38b4abebf8fe682971a1422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "839d7002c8844d93b77682ee8e2204a4" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 43645 } } peers { permanent_uuid: "48180e07e38b4abebf8fe682971a1422" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 37129 } } peers { permanent_uuid: "c76cd0b6fb8b4790ad3264dff35ec16b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45555 } }
I20260808 06:08:44.892380 1303 raft_consensus.cc:374] T c5fa0f3f2a6a4ebaa353868230609749 P 48180e07e38b4abebf8fe682971a1422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:08:44.892434 1303 raft_consensus.cc:729] T c5fa0f3f2a6a4ebaa353868230609749 P 48180e07e38b4abebf8fe682971a1422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48180e07e38b4abebf8fe682971a1422, State: Initialized, Role: FOLLOWER
I20260808 06:08:44.893074 1303 consensus_queue.cc:260] T c5fa0f3f2a6a4ebaa353868230609749 P 48180e07e38b4abebf8fe682971a1422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "839d7002c8844d93b77682ee8e2204a4" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 43645 } } peers { permanent_uuid: "48180e07e38b4abebf8fe682971a1422" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 37129 } } peers { permanent_uuid: "c76cd0b6fb8b4790ad3264dff35ec16b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45555 } }
I20260808 06:08:44.893678 1281 heartbeater.cc:503] Master 127.1.151.190:43305 was elected leader, sending a full tablet report...
I20260808 06:08:44.893893 1303 ts_tablet_manager.cc:1428] T c5fa0f3f2a6a4ebaa353868230609749 P 48180e07e38b4abebf8fe682971a1422: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260808 06:08:44.897644 1065 tablet.cc:2369] T c5fa0f3f2a6a4ebaa353868230609749 P c76cd0b6fb8b4790ad3264dff35ec16b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 06:08:44.928248 1311 raft_consensus.cc:482] T c5fa0f3f2a6a4ebaa353868230609749 P 48180e07e38b4abebf8fe682971a1422 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 06:08:44.928421 1311 raft_consensus.cc:504] T c5fa0f3f2a6a4ebaa353868230609749 P 48180e07e38b4abebf8fe682971a1422 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "839d7002c8844d93b77682ee8e2204a4" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 43645 } } peers { permanent_uuid: "48180e07e38b4abebf8fe682971a1422" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 37129 } } peers { permanent_uuid: "c76cd0b6fb8b4790ad3264dff35ec16b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45555 } }
I20260808 06:08:44.929463 1311 leader_election.cc:288] T c5fa0f3f2a6a4ebaa353868230609749 P 48180e07e38b4abebf8fe682971a1422 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 839d7002c8844d93b77682ee8e2204a4 (127.1.151.129:43645), c76cd0b6fb8b4790ad3264dff35ec16b (127.1.151.130:45555)
I20260808 06:08:44.932899 1283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:44.929673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:43645 (local address 127.1.151.131:53667)
0808 06:08:44.929800 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:44.929817 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:44.929908 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:44.930174 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:44.930177 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:44.930201 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:08:44.930332 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:44.930340 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.931068 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.931071 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:44.931865 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:44.931873 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.932493 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.932497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:44.932595 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:44.932618 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 06:08:44.932695 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 06:08:44.932767 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":1}
I20260808 06:08:44.932898 1296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:44.929797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:53667 (local address 127.1.151.129:43645)
0808 06:08:44.929903 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:44.929906 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:44.929921 (+ 15us) server_negotiation.cc:407] Connection header received
0808 06:08:44.929997 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:44.930000 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:44.930050 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:44.930129 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:08:44.930434 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.930961 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.931993 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.932385 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.932511 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:44.932564 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:44.932726 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 06:08:44.932795 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 06:08:44.933318 1315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:44.929974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.130:45555 (local address 127.1.151.131:40167)
0808 06:08:44.930413 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:44.930426 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 06:08:44.930515 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:08:44.930720 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:08:44.930723 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:08:44.930738 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 06:08:44.930843 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:44.930848 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.931521 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.931524 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:44.932267 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:08:44.932275 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.932883 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.932886 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:08:44.932980 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:44.933005 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 06:08:44.933108 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 06:08:44.933170 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":110,"threads_started":1}
I20260808 06:08:44.933341 1299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:44.930083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.151.131:40167 (local address 127.1.151.130:45555)
0808 06:08:44.930230 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:44.930235 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:44.930485 (+ 250us) server_negotiation.cc:407] Connection header received
0808 06:08:44.930592 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:44.930595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:44.930639 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:44.930698 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 06:08:44.930936 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.931418 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.932380 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.932765 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.932866 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:44.932912 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:44.933152 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 06:08:44.933233 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 06:08:44.933533 790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5fa0f3f2a6a4ebaa353868230609749" candidate_uuid: "48180e07e38b4abebf8fe682971a1422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "839d7002c8844d93b77682ee8e2204a4" is_pre_election: true
I20260808 06:08:44.933830 790 raft_consensus.cc:2454] T c5fa0f3f2a6a4ebaa353868230609749 P 839d7002c8844d93b77682ee8e2204a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48180e07e38b4abebf8fe682971a1422 in term 0.
I20260808 06:08:44.933979 994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5fa0f3f2a6a4ebaa353868230609749" candidate_uuid: "48180e07e38b4abebf8fe682971a1422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c76cd0b6fb8b4790ad3264dff35ec16b" is_pre_election: true
I20260808 06:08:44.934271 994 raft_consensus.cc:2454] T c5fa0f3f2a6a4ebaa353868230609749 P c76cd0b6fb8b4790ad3264dff35ec16b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48180e07e38b4abebf8fe682971a1422 in term 0.
I20260808 06:08:44.934330 1094 leader_election.cc:302] T c5fa0f3f2a6a4ebaa353868230609749 P 48180e07e38b4abebf8fe682971a1422 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48180e07e38b4abebf8fe682971a1422, 839d7002c8844d93b77682ee8e2204a4; no voters:
I20260808 06:08:44.934587 1311 raft_consensus.cc:2781] T c5fa0f3f2a6a4ebaa353868230609749 P 48180e07e38b4abebf8fe682971a1422 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 06:08:44.934665 1311 raft_consensus.cc:482] T c5fa0f3f2a6a4ebaa353868230609749 P 48180e07e38b4abebf8fe682971a1422 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 06:08:44.934696 1311 raft_consensus.cc:3037] T c5fa0f3f2a6a4ebaa353868230609749 P 48180e07e38b4abebf8fe682971a1422 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:08:44.935700 1311 raft_consensus.cc:504] T c5fa0f3f2a6a4ebaa353868230609749 P 48180e07e38b4abebf8fe682971a1422 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "839d7002c8844d93b77682ee8e2204a4" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 43645 } } peers { permanent_uuid: "48180e07e38b4abebf8fe682971a1422" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 37129 } } peers { permanent_uuid: "c76cd0b6fb8b4790ad3264dff35ec16b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45555 } }
I20260808 06:08:44.936007 1311 leader_election.cc:288] T c5fa0f3f2a6a4ebaa353868230609749 P 48180e07e38b4abebf8fe682971a1422 [CANDIDATE]: Term 1 election: Requested vote from peers 839d7002c8844d93b77682ee8e2204a4 (127.1.151.129:43645), c76cd0b6fb8b4790ad3264dff35ec16b (127.1.151.130:45555)
I20260808 06:08:44.936369 790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5fa0f3f2a6a4ebaa353868230609749" candidate_uuid: "48180e07e38b4abebf8fe682971a1422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "839d7002c8844d93b77682ee8e2204a4"
I20260808 06:08:44.936405 994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5fa0f3f2a6a4ebaa353868230609749" candidate_uuid: "48180e07e38b4abebf8fe682971a1422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c76cd0b6fb8b4790ad3264dff35ec16b"
I20260808 06:08:44.936484 790 raft_consensus.cc:3037] T c5fa0f3f2a6a4ebaa353868230609749 P 839d7002c8844d93b77682ee8e2204a4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:08:44.936537 994 raft_consensus.cc:3037] T c5fa0f3f2a6a4ebaa353868230609749 P c76cd0b6fb8b4790ad3264dff35ec16b [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:08:44.937500 994 raft_consensus.cc:2454] T c5fa0f3f2a6a4ebaa353868230609749 P c76cd0b6fb8b4790ad3264dff35ec16b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48180e07e38b4abebf8fe682971a1422 in term 1.
I20260808 06:08:44.937501 790 raft_consensus.cc:2454] T c5fa0f3f2a6a4ebaa353868230609749 P 839d7002c8844d93b77682ee8e2204a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48180e07e38b4abebf8fe682971a1422 in term 1.
I20260808 06:08:44.937773 1094 leader_election.cc:302] T c5fa0f3f2a6a4ebaa353868230609749 P 48180e07e38b4abebf8fe682971a1422 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48180e07e38b4abebf8fe682971a1422, 839d7002c8844d93b77682ee8e2204a4; no voters:
I20260808 06:08:44.937929 1311 raft_consensus.cc:2781] T c5fa0f3f2a6a4ebaa353868230609749 P 48180e07e38b4abebf8fe682971a1422 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:08:44.938164 1311 raft_consensus.cc:686] T c5fa0f3f2a6a4ebaa353868230609749 P 48180e07e38b4abebf8fe682971a1422 [term 1 LEADER]: Becoming Leader. State: Replica: 48180e07e38b4abebf8fe682971a1422, State: Running, Role: LEADER
I20260808 06:08:44.938468 1311 consensus_queue.cc:237] T c5fa0f3f2a6a4ebaa353868230609749 P 48180e07e38b4abebf8fe682971a1422 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "839d7002c8844d93b77682ee8e2204a4" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 43645 } } peers { permanent_uuid: "48180e07e38b4abebf8fe682971a1422" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 37129 } } peers { permanent_uuid: "c76cd0b6fb8b4790ad3264dff35ec16b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45555 } }
I20260808 06:08:44.941749 567 catalog_manager.cc:5495] T c5fa0f3f2a6a4ebaa353868230609749 P 48180e07e38b4abebf8fe682971a1422 reported cstate change: term changed from 0 to 1, leader changed from <none> to 48180e07e38b4abebf8fe682971a1422 (127.1.151.131). New cstate: current_term: 1 leader_uuid: "48180e07e38b4abebf8fe682971a1422" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "839d7002c8844d93b77682ee8e2204a4" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 43645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48180e07e38b4abebf8fe682971a1422" member_type: VOTER last_known_addr { host: "127.1.151.131" port: 37129 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c76cd0b6fb8b4790ad3264dff35ec16b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45555 } health_report { overall_health: UNKNOWN } } }
W20260808 06:08:44.946399 866 tablet.cc:2369] T c5fa0f3f2a6a4ebaa353868230609749 P 839d7002c8844d93b77682ee8e2204a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 06:08:44.966794 1298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:44.963261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46376 (local address 127.1.151.131:37129)
0808 06:08:44.963414 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:44.963418 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:44.963450 (+ 32us) server_negotiation.cc:407] Connection header received
0808 06:08:44.963525 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:44.963527 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:44.963591 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:44.963677 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:08:44.963935 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.964474 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.965331 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:44.965561 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:44.965657 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:44.965949 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:08:44.966395 (+ 446us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:08:44.966502 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:44.966586 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 06:08:44.966644 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 06:08:44.972348 994 raft_consensus.cc:1261] T c5fa0f3f2a6a4ebaa353868230609749 P c76cd0b6fb8b4790ad3264dff35ec16b [term 1 FOLLOWER]: Refusing update from remote peer 48180e07e38b4abebf8fe682971a1422: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:08:44.972348 790 raft_consensus.cc:1261] T c5fa0f3f2a6a4ebaa353868230609749 P 839d7002c8844d93b77682ee8e2204a4 [term 1 FOLLOWER]: Refusing update from remote peer 48180e07e38b4abebf8fe682971a1422: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:08:44.973006 1311 consensus_queue.cc:1035] T c5fa0f3f2a6a4ebaa353868230609749 P 48180e07e38b4abebf8fe682971a1422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c76cd0b6fb8b4790ad3264dff35ec16b" member_type: VOTER last_known_addr { host: "127.1.151.130" port: 45555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:08:44.973201 1316 consensus_queue.cc:1035] T c5fa0f3f2a6a4ebaa353868230609749 P 48180e07e38b4abebf8fe682971a1422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "839d7002c8844d93b77682ee8e2204a4" member_type: VOTER last_known_addr { host: "127.1.151.129" port: 43645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:08:44.987617 567 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:08:44.987811 567 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 06:08:44.989919 567 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 88ff74125f85430dae0b676fc752c90b: Sending DeleteTablet for 3 replicas of tablet c5fa0f3f2a6a4ebaa353868230609749
I20260808 06:08:44.990821 1169 tablet_service.cc:1514] Processing DeleteTablet for tablet c5fa0f3f2a6a4ebaa353868230609749 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:08:44 UTC) from {username='slave'} at 127.0.0.1:46374
I20260808 06:08:44.990823 967 tablet_service.cc:1514] Processing DeleteTablet for tablet c5fa0f3f2a6a4ebaa353868230609749 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:08:44 UTC) from {username='slave'} at 127.0.0.1:43194
I20260808 06:08:44.991214 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 663
I20260808 06:08:44.991317 762 tablet_service.cc:1514] Processing DeleteTablet for tablet c5fa0f3f2a6a4ebaa353868230609749 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 06:08:44 UTC) from {username='slave'} at 127.0.0.1:52110
I20260808 06:08:44.996693 1334 tablet_replica.cc:330] T c5fa0f3f2a6a4ebaa353868230609749 P 48180e07e38b4abebf8fe682971a1422: stopping tablet replica
I20260808 06:08:44.997100 1334 raft_consensus.cc:2229] T c5fa0f3f2a6a4ebaa353868230609749 P 48180e07e38b4abebf8fe682971a1422 [term 1 LEADER]: Raft consensus shutting down.
I20260808 06:08:44.997602 1334 raft_consensus.cc:2258] T c5fa0f3f2a6a4ebaa353868230609749 P 48180e07e38b4abebf8fe682971a1422 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:08:45.000321 1335 tablet_replica.cc:330] T c5fa0f3f2a6a4ebaa353868230609749 P c76cd0b6fb8b4790ad3264dff35ec16b: stopping tablet replica
I20260808 06:08:45.000679 1335 raft_consensus.cc:2229] T c5fa0f3f2a6a4ebaa353868230609749 P c76cd0b6fb8b4790ad3264dff35ec16b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 06:08:45.000973 1335 raft_consensus.cc:2258] T c5fa0f3f2a6a4ebaa353868230609749 P c76cd0b6fb8b4790ad3264dff35ec16b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 06:08:45.002434 1334 ts_tablet_manager.cc:1905] T c5fa0f3f2a6a4ebaa353868230609749 P 48180e07e38b4abebf8fe682971a1422: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:08:45.002458 1335 ts_tablet_manager.cc:1905] T c5fa0f3f2a6a4ebaa353868230609749 P c76cd0b6fb8b4790ad3264dff35ec16b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 06:08:45.005206 1334 ts_tablet_manager.cc:1918] T c5fa0f3f2a6a4ebaa353868230609749 P 48180e07e38b4abebf8fe682971a1422: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:08:45.005367 1334 log.cc:1196] T c5fa0f3f2a6a4ebaa353868230609749 P 48180e07e38b4abebf8fe682971a1422: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-2/wal/wals/c5fa0f3f2a6a4ebaa353868230609749
I20260808 06:08:45.005599 1335 ts_tablet_manager.cc:1918] T c5fa0f3f2a6a4ebaa353868230609749 P c76cd0b6fb8b4790ad3264dff35ec16b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 06:08:45.005702 1335 log.cc:1196] T c5fa0f3f2a6a4ebaa353868230609749 P c76cd0b6fb8b4790ad3264dff35ec16b: Deleting WAL directory at /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786169070948234-1630-0/minicluster-data/ts-1/wal/wals/c5fa0f3f2a6a4ebaa353868230609749
I20260808 06:08:45.005879 1334 ts_tablet_manager.cc:1939] T c5fa0f3f2a6a4ebaa353868230609749 P 48180e07e38b4abebf8fe682971a1422: Deleting consensus metadata
I20260808 06:08:45.005998 1335 ts_tablet_manager.cc:1939] T c5fa0f3f2a6a4ebaa353868230609749 P c76cd0b6fb8b4790ad3264dff35ec16b: Deleting consensus metadata
I20260808 06:08:45.007119 545 catalog_manager.cc:4841] TS 48180e07e38b4abebf8fe682971a1422 (127.1.151.131:37129): tablet c5fa0f3f2a6a4ebaa353868230609749 (table test-table [id=871dbeeb9cb14f7094b0652590b6da77]) successfully deleted
I20260808 06:08:45.007395 547 catalog_manager.cc:4841] TS c76cd0b6fb8b4790ad3264dff35ec16b (127.1.151.130:45555): tablet c5fa0f3f2a6a4ebaa353868230609749 (table test-table [id=871dbeeb9cb14f7094b0652590b6da77]) successfully deleted
W20260808 06:08:45.013549 547 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.151.129:43645 (error 108)
I20260808 06:08:45.013775 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 872
I20260808 06:08:45.023154 1297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:45.022834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.129:43645 (local address 127.0.0.1:52118)
0808 06:08:45.022976 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 06:08:45.023052 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.129:43645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 06:08:45.023442 547 catalog_manager.cc:4584] TS 839d7002c8844d93b77682ee8e2204a4 (127.1.151.129:43645): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c5fa0f3f2a6a4ebaa353868230609749: Network error: Client connection negotiation failed: client connection to 127.1.151.129:43645: connect: Connection refused (error 111)
I20260808 06:08:45.024833 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 1068
I20260808 06:08:45.035347 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 524
2026-08-08T06:08:45Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1024 ms)
[----------] 6 tests from EncryptionPolicyTest (10820 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:08:45 dist-test-slave-zj8q krb5kdc[1342](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:08:45 dist-test-slave-zj8q krb5kdc[1342](info): set up 2 sockets
Aug 08 06:08:45 dist-test-slave-zj8q krb5kdc[1342](info): commencing operation
krb5kdc: starting...
W20260808 06:08:46.173856 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:08:46 dist-test-slave-zj8q krb5kdc[1342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:08:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:08:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 06:08:46.295042 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38287
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38287
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:08:46.404582 1360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:46.404820 1360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:46.404865 1360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:46.408751 1360 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:08:46.408802 1360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:46.408819 1360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:46.408838 1360 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:08:46.408866 1360 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:08:46.413921 1360 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38287
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:46.414912 1360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:46.415882 1360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:08:46.424015 1372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:46.424091 1360 server_base.cc:1034] running on GCE node
W20260808 06:08:46.424015 1370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:46.424043 1376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:46.424753 1360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:46.425789 1360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:46.426949 1360 hybrid_clock.cc:648] HybridClock initialized: now 1786169326426925 us; error 34 us; skew 500 ppm
Aug 08 06:08:46 dist-test-slave-zj8q krb5kdc[1342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169326, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:08:46.429569 1360 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 06:08:46.430495 1360 webserver.cc:460] Webserver started at http://127.0.0.1:38621/ using document root <none> and password file <none>
I20260808 06:08:46.431015 1360 fs_manager.cc:359] Metadata directory not provided
I20260808 06:08:46.431089 1360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:08:46.431283 1360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:08:46.433224 1360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "c00b13f817e04032a324deee02d3ceda"
format_stamp: "Formatted at 2026-08-08 06:08:46 on dist-test-slave-zj8q"
server_key: "93ebd0fa06ea264e2a4950727d52552c"
server_key_iv: "6f202c1b3a2ac343dd0baa51d6155bd4"
server_key_version: "encryptionkey@0"
I20260808 06:08:46.433786 1360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "c00b13f817e04032a324deee02d3ceda"
format_stamp: "Formatted at 2026-08-08 06:08:46 on dist-test-slave-zj8q"
server_key: "93ebd0fa06ea264e2a4950727d52552c"
server_key_iv: "6f202c1b3a2ac343dd0baa51d6155bd4"
server_key_version: "encryptionkey@0"
I20260808 06:08:46.437296 1360 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 06:08:46.439630 1388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:46.440743 1360 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 06:08:46.440853 1360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "c00b13f817e04032a324deee02d3ceda"
format_stamp: "Formatted at 2026-08-08 06:08:46 on dist-test-slave-zj8q"
server_key: "93ebd0fa06ea264e2a4950727d52552c"
server_key_iv: "6f202c1b3a2ac343dd0baa51d6155bd4"
server_key_version: "encryptionkey@0"
I20260808 06:08:46.440943 1360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:08:46.462802 1360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:08:46.465459 1360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:08:46.465651 1360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:08:46.472410 1360 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38287
I20260808 06:08:46.472432 1465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38287 every 8 connection(s)
I20260808 06:08:46.473282 1360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:08:46.476019 1466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:08:46.480815 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 1360
I20260808 06:08:46.480942 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:08:46.481237 1630 external_mini_cluster.cc:1428] Setting key b9c1fad02cc00c6400637a5857787f06
I20260808 06:08:46.481575 1466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c00b13f817e04032a324deee02d3ceda: Bootstrap starting.
I20260808 06:08:46.484207 1466 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c00b13f817e04032a324deee02d3ceda: Neither blocks nor log segments found. Creating new log.
I20260808 06:08:46.485081 1466 log.cc:824] T 00000000000000000000000000000000 P c00b13f817e04032a324deee02d3ceda: Log is configured to *not* fsync() on all Append() calls
I20260808 06:08:46.487227 1466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c00b13f817e04032a324deee02d3ceda: No bootstrap required, opened a new log
Aug 08 06:08:46 dist-test-slave-zj8q krb5kdc[1342](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 06:08:46.491331 1466 raft_consensus.cc:348] T 00000000000000000000000000000000 P c00b13f817e04032a324deee02d3ceda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c00b13f817e04032a324deee02d3ceda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38287 } }
I20260808 06:08:46.491564 1466 raft_consensus.cc:374] T 00000000000000000000000000000000 P c00b13f817e04032a324deee02d3ceda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:08:46.491611 1466 raft_consensus.cc:729] T 00000000000000000000000000000000 P c00b13f817e04032a324deee02d3ceda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c00b13f817e04032a324deee02d3ceda, State: Initialized, Role: FOLLOWER
I20260808 06:08:46.492058 1466 consensus_queue.cc:260] T 00000000000000000000000000000000 P c00b13f817e04032a324deee02d3ceda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c00b13f817e04032a324deee02d3ceda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38287 } }
I20260808 06:08:46.492244 1466 raft_consensus.cc:388] T 00000000000000000000000000000000 P c00b13f817e04032a324deee02d3ceda [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:08:46.492316 1466 raft_consensus.cc:482] T 00000000000000000000000000000000 P c00b13f817e04032a324deee02d3ceda [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:08:46.492412 1466 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c00b13f817e04032a324deee02d3ceda [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:08:46.493532 1466 raft_consensus.cc:504] T 00000000000000000000000000000000 P c00b13f817e04032a324deee02d3ceda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c00b13f817e04032a324deee02d3ceda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38287 } }
I20260808 06:08:46.493875 1466 leader_election.cc:302] T 00000000000000000000000000000000 P c00b13f817e04032a324deee02d3ceda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c00b13f817e04032a324deee02d3ceda; no voters:
I20260808 06:08:46.494251 1466 leader_election.cc:288] T 00000000000000000000000000000000 P c00b13f817e04032a324deee02d3ceda [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:08:46.494374 1476 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c00b13f817e04032a324deee02d3ceda [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:08:46.494594 1476 raft_consensus.cc:686] T 00000000000000000000000000000000 P c00b13f817e04032a324deee02d3ceda [term 1 LEADER]: Becoming Leader. State: Replica: c00b13f817e04032a324deee02d3ceda, State: Running, Role: LEADER
I20260808 06:08:46.494925 1476 consensus_queue.cc:237] T 00000000000000000000000000000000 P c00b13f817e04032a324deee02d3ceda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c00b13f817e04032a324deee02d3ceda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38287 } }
I20260808 06:08:46.495572 1466 sys_catalog.cc:564] T 00000000000000000000000000000000 P c00b13f817e04032a324deee02d3ceda [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:08:46.496731 1477 sys_catalog.cc:455] T 00000000000000000000000000000000 P c00b13f817e04032a324deee02d3ceda [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c00b13f817e04032a324deee02d3ceda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c00b13f817e04032a324deee02d3ceda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38287 } } }
I20260808 06:08:46.496731 1479 sys_catalog.cc:455] T 00000000000000000000000000000000 P c00b13f817e04032a324deee02d3ceda [sys.catalog]: SysCatalogTable state changed. Reason: New leader c00b13f817e04032a324deee02d3ceda. Latest consensus state: current_term: 1 leader_uuid: "c00b13f817e04032a324deee02d3ceda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c00b13f817e04032a324deee02d3ceda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38287 } } }
I20260808 06:08:46.496876 1479 sys_catalog.cc:458] T 00000000000000000000000000000000 P c00b13f817e04032a324deee02d3ceda [sys.catalog]: This master's current role is: LEADER
I20260808 06:08:46.496876 1477 sys_catalog.cc:458] T 00000000000000000000000000000000 P c00b13f817e04032a324deee02d3ceda [sys.catalog]: This master's current role is: LEADER
I20260808 06:08:46.497138 1473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:46.482641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48663 (local address 127.0.0.1:38287)
0808 06:08:46.483062 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:46.483080 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:46.483122 (+ 42us) server_negotiation.cc:407] Connection header received
0808 06:08:46.483917 (+ 795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:46.483942 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:46.484365 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:46.484731 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:46.485385 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:46.486750 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:46.487671 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:46.488345 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:46.488517 (+ 172us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:46.491221 (+ 2704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:46.491227 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:46.491240 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:08:46.491268 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:46.493420 (+ 2152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:46.493977 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:46.493984 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:46.493989 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:46.494130 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:46.494564 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:46.494568 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:46.494569 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:46.495123 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:46.495343 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:46.495703 (+ 360us) server_negotiation.cc:299] Negotiation successful
0808 06:08:46.496131 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":113,"threads_started":1}
I20260808 06:08:46.497290 1488 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:08:46.500594 1630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 06:08:46.500679 1488 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:08:46.506836 1488 catalog_manager.cc:1347] Generated new cluster ID: c6a135d115094e84b198ef13ac17d115
I20260808 06:08:46.506903 1488 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:08:46.514324 1473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:46.502565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33921 (local address 127.0.0.1:38287)
0808 06:08:46.502670 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:46.502674 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:46.502863 (+ 189us) server_negotiation.cc:407] Connection header received
0808 06:08:46.502974 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:46.502977 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:46.503021 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:46.503099 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:46.503879 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:46.504627 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:46.505781 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:46.506115 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:46.506248 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:46.511271 (+ 5023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:46.511275 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:46.511277 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:08:46.511299 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:46.512621 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:46.513086 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:46.513088 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:46.513090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:46.513171 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:46.513480 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:46.513484 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:46.513485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:46.513660 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:46.513793 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:46.514061 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 06:08:46.514188 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 06:08:46.532678 1488 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:08:46.533598 1488 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:08:46.540693 1488 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c00b13f817e04032a324deee02d3ceda: Generated new TSK 0
I20260808 06:08:46.541361 1488 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 06:08:46.548485 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 1360
W20260808 06:08:46.560374 1339 connection.cc:535] client connection to 127.0.0.1:38287 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38287): Connection reset by peer (error 104)
2026-08-08T06:08:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1509 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:08:46 dist-test-slave-zj8q krb5kdc[1523](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:08:46 dist-test-slave-zj8q krb5kdc[1523](info): set up 2 sockets
Aug 08 06:08:46 dist-test-slave-zj8q krb5kdc[1523](info): commencing operation
krb5kdc: starting...
W20260808 06:08:47.698748 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:08:47 dist-test-slave-zj8q krb5kdc[1523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:08:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:08:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 06:08:47.816486 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41963
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41963
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:08:47.923631 1538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:47.923874 1538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:47.923930 1538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:47.927834 1538 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:08:47.927884 1538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:47.927911 1538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:47.927932 1538 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:08:47.927961 1538 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:08:47.932999 1538 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41963
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:47.933930 1538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:47.934886 1538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:08:47.942850 1552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:47.942875 1547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:47.942876 1549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:47.943014 1538 server_base.cc:1034] running on GCE node
I20260808 06:08:47.943648 1538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:47.944674 1538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:47.945851 1538 hybrid_clock.cc:648] HybridClock initialized: now 1786169327945831 us; error 29 us; skew 500 ppm
Aug 08 06:08:47 dist-test-slave-zj8q krb5kdc[1523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169327, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:08:47.948628 1538 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 06:08:47.949478 1538 webserver.cc:460] Webserver started at http://127.0.0.1:42545/ using document root <none> and password file <none>
I20260808 06:08:47.949997 1538 fs_manager.cc:359] Metadata directory not provided
I20260808 06:08:47.950066 1538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:08:47.950290 1538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:08:47.952313 1538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "92b34bac8fd54f5eb934e33b083273da"
format_stamp: "Formatted at 2026-08-08 06:08:47 on dist-test-slave-zj8q"
server_key: "8894af408c462c79d094cf24c342ad14"
server_key_iv: "ffa6d23c08498fba5f07659fa8eb5fc2"
server_key_version: "encryptionkey@0"
I20260808 06:08:47.952872 1538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "92b34bac8fd54f5eb934e33b083273da"
format_stamp: "Formatted at 2026-08-08 06:08:47 on dist-test-slave-zj8q"
server_key: "8894af408c462c79d094cf24c342ad14"
server_key_iv: "ffa6d23c08498fba5f07659fa8eb5fc2"
server_key_version: "encryptionkey@0"
I20260808 06:08:47.956478 1538 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 06:08:47.958859 1561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:47.959981 1538 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:08:47.960093 1538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "92b34bac8fd54f5eb934e33b083273da"
format_stamp: "Formatted at 2026-08-08 06:08:47 on dist-test-slave-zj8q"
server_key: "8894af408c462c79d094cf24c342ad14"
server_key_iv: "ffa6d23c08498fba5f07659fa8eb5fc2"
server_key_version: "encryptionkey@0"
I20260808 06:08:47.960187 1538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:08:47.983651 1538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:08:47.986670 1538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:08:47.986840 1538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:08:47.993501 1538 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41963
I20260808 06:08:47.993525 1645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41963 every 8 connection(s)
I20260808 06:08:47.994426 1538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:08:47.997120 1647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:08:48.002140 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 1538
I20260808 06:08:48.002305 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:08:48.002645 1630 external_mini_cluster.cc:1428] Setting key a2be856aa66c0653fabee50ee968873e
I20260808 06:08:48.002709 1647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92b34bac8fd54f5eb934e33b083273da: Bootstrap starting.
I20260808 06:08:48.005201 1647 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92b34bac8fd54f5eb934e33b083273da: Neither blocks nor log segments found. Creating new log.
I20260808 06:08:48.005915 1647 log.cc:824] T 00000000000000000000000000000000 P 92b34bac8fd54f5eb934e33b083273da: Log is configured to *not* fsync() on all Append() calls
I20260808 06:08:48.007814 1647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92b34bac8fd54f5eb934e33b083273da: No bootstrap required, opened a new log
I20260808 06:08:48.011125 1650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:48.003985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56907 (local address 127.0.0.1:41963)
0808 06:08:48.004422 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:48.004437 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:48.004470 (+ 33us) server_negotiation.cc:407] Connection header received
0808 06:08:48.005092 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:48.005111 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:48.005453 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:48.005766 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:48.006183 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:48.007318 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:48.008200 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:48.008664 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:48.008808 (+ 144us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:48.009584 (+ 776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:48.009589 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:48.009608 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:08:48.009960 (+ 352us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:48.010012 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:08:48.010033 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:48.010145 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:48.010350 (+ 205us) server_negotiation.cc:299] Negotiation successful
0808 06:08:48.010474 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":140,"threads_started":1}
I20260808 06:08:48.011665 1647 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92b34bac8fd54f5eb934e33b083273da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92b34bac8fd54f5eb934e33b083273da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41963 } }
I20260808 06:08:48.011924 1647 raft_consensus.cc:374] T 00000000000000000000000000000000 P 92b34bac8fd54f5eb934e33b083273da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:08:48.011978 1647 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92b34bac8fd54f5eb934e33b083273da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92b34bac8fd54f5eb934e33b083273da, State: Initialized, Role: FOLLOWER
I20260808 06:08:48.012373 1647 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92b34bac8fd54f5eb934e33b083273da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92b34bac8fd54f5eb934e33b083273da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41963 } }
I20260808 06:08:48.014029 1630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 06:08:48.015889 1647 raft_consensus.cc:388] T 00000000000000000000000000000000 P 92b34bac8fd54f5eb934e33b083273da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:08:48.015990 1647 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92b34bac8fd54f5eb934e33b083273da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:08:48.016103 1647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92b34bac8fd54f5eb934e33b083273da [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:08:48.017248 1647 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92b34bac8fd54f5eb934e33b083273da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92b34bac8fd54f5eb934e33b083273da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41963 } }
I20260808 06:08:48.017628 1647 leader_election.cc:302] T 00000000000000000000000000000000 P 92b34bac8fd54f5eb934e33b083273da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92b34bac8fd54f5eb934e33b083273da; no voters:
I20260808 06:08:48.017961 1647 leader_election.cc:288] T 00000000000000000000000000000000 P 92b34bac8fd54f5eb934e33b083273da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:08:48.018316 1660 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92b34bac8fd54f5eb934e33b083273da [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:08:48.018561 1660 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92b34bac8fd54f5eb934e33b083273da [term 1 LEADER]: Becoming Leader. State: Replica: 92b34bac8fd54f5eb934e33b083273da, State: Running, Role: LEADER
I20260808 06:08:48.018901 1660 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92b34bac8fd54f5eb934e33b083273da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92b34bac8fd54f5eb934e33b083273da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41963 } }
I20260808 06:08:48.020177 1660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92b34bac8fd54f5eb934e33b083273da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "92b34bac8fd54f5eb934e33b083273da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92b34bac8fd54f5eb934e33b083273da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41963 } } }
I20260808 06:08:48.020256 1659 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92b34bac8fd54f5eb934e33b083273da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92b34bac8fd54f5eb934e33b083273da. Latest consensus state: current_term: 1 leader_uuid: "92b34bac8fd54f5eb934e33b083273da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92b34bac8fd54f5eb934e33b083273da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41963 } } }
I20260808 06:08:48.020356 1660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92b34bac8fd54f5eb934e33b083273da [sys.catalog]: This master's current role is: LEADER
I20260808 06:08:48.020663 1647 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92b34bac8fd54f5eb934e33b083273da [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:08:48.020925 1650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:48.016165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41705 (local address 127.0.0.1:41963)
0808 06:08:48.016289 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:48.016293 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:48.016659 (+ 366us) server_negotiation.cc:407] Connection header received
0808 06:08:48.016792 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:48.016795 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:48.016833 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:48.016911 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:48.017227 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:48.017862 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:48.018986 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:48.019326 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:48.019399 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:48.020442 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:48.020446 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:48.020449 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:08:48.020461 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:48.020485 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:08:48.020495 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:48.020539 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:48.020796 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 06:08:48.020829 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 06:08:48.020543 1659 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92b34bac8fd54f5eb934e33b083273da [sys.catalog]: This master's current role is: LEADER
I20260808 06:08:48.022197 1666 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:08:48.025449 1666 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:08:48.030917 1666 catalog_manager.cc:1347] Generated new cluster ID: 6d85ac6c246d41e9a6014aec9c327682
I20260808 06:08:48.030994 1666 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:08:48.040839 1666 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:08:48.041677 1666 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:08:48.049824 1666 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 92b34bac8fd54f5eb934e33b083273da: Generated new TSK 0
I20260808 06:08:48.050467 1666 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 06:08:48.054975 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 1538
W20260808 06:08:48.066336 1517 connection.cc:535] client connection to 127.0.0.1:41963 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41963): Connection reset by peer (error 104)
2026-08-08T06:08:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1506 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:08:48 dist-test-slave-zj8q krb5kdc[1692](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:08:48 dist-test-slave-zj8q krb5kdc[1692](info): set up 2 sockets
Aug 08 06:08:48 dist-test-slave-zj8q krb5kdc[1692](info): commencing operation
krb5kdc: starting...
W20260808 06:08:49.197273 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:08:49 dist-test-slave-zj8q krb5kdc[1692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:08:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:08:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 06:08:49.317008 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38099
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38099
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:08:49.425555 1723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:49.425803 1723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:49.425864 1723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:49.429817 1723 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:08:49.429880 1723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:49.429906 1723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:49.429927 1723 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:08:49.429955 1723 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:08:49.435034 1723 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38099
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:49.435956 1723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:49.436908 1723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:08:49.444984 1734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:49.445142 1723 server_base.cc:1034] running on GCE node
W20260808 06:08:49.445003 1732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:49.444986 1730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:49.445859 1723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:49.446894 1723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:49.448062 1723 hybrid_clock.cc:648] HybridClock initialized: now 1786169329448044 us; error 31 us; skew 500 ppm
Aug 08 06:08:49 dist-test-slave-zj8q krb5kdc[1692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169329, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:08:49.450731 1723 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 06:08:49.451558 1723 webserver.cc:460] Webserver started at http://127.0.0.1:41075/ using document root <none> and password file <none>
I20260808 06:08:49.452070 1723 fs_manager.cc:359] Metadata directory not provided
I20260808 06:08:49.452137 1723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:08:49.452332 1723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:08:49.454264 1723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "7df1629fa3634306b5eeca2c74239018"
format_stamp: "Formatted at 2026-08-08 06:08:49 on dist-test-slave-zj8q"
server_key: "8f14f51216fe9e0926fdefdb2ad308f0"
server_key_iv: "d75b6ddc305418507960077f7b53761a"
server_key_version: "encryptionkey@0"
I20260808 06:08:49.454805 1723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "7df1629fa3634306b5eeca2c74239018"
format_stamp: "Formatted at 2026-08-08 06:08:49 on dist-test-slave-zj8q"
server_key: "8f14f51216fe9e0926fdefdb2ad308f0"
server_key_iv: "d75b6ddc305418507960077f7b53761a"
server_key_version: "encryptionkey@0"
I20260808 06:08:49.458247 1723 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 06:08:49.460498 1741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:49.461658 1723 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:08:49.461781 1723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "7df1629fa3634306b5eeca2c74239018"
format_stamp: "Formatted at 2026-08-08 06:08:49 on dist-test-slave-zj8q"
server_key: "8f14f51216fe9e0926fdefdb2ad308f0"
server_key_iv: "d75b6ddc305418507960077f7b53761a"
server_key_version: "encryptionkey@0"
I20260808 06:08:49.461871 1723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:08:49.466248 1723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:08:49.468971 1723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:08:49.469131 1723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:08:49.475968 1723 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38099
I20260808 06:08:49.475986 1822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38099 every 8 connection(s)
I20260808 06:08:49.476857 1723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:08:49.479611 1825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:08:49.482824 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 1723
I20260808 06:08:49.482967 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:08:49.483292 1630 external_mini_cluster.cc:1428] Setting key a53edf383cd4b4230cd7c5f100f922da
I20260808 06:08:49.485555 1825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7df1629fa3634306b5eeca2c74239018: Bootstrap starting.
I20260808 06:08:49.488083 1825 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7df1629fa3634306b5eeca2c74239018: Neither blocks nor log segments found. Creating new log.
I20260808 06:08:49.488479 1828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:49.484576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44517 (local address 127.0.0.1:38099)
0808 06:08:49.485120 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:49.485134 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:49.485169 (+ 35us) server_negotiation.cc:407] Connection header received
0808 06:08:49.485817 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:49.485837 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:49.486118 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:49.486431 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:49.487263 (+ 832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:49.487268 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:49.487280 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:08:49.487291 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:49.487348 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:08:49.487368 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:49.487491 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:49.487606 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 06:08:49.487750 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":115,"threads_started":1}
I20260808 06:08:49.488833 1825 log.cc:824] T 00000000000000000000000000000000 P 7df1629fa3634306b5eeca2c74239018: Log is configured to *not* fsync() on all Append() calls
I20260808 06:08:49.490942 1825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7df1629fa3634306b5eeca2c74239018: No bootstrap required, opened a new log
I20260808 06:08:49.491372 1630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 06:08:49.495217 1828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:49.493703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49623 (local address 127.0.0.1:38099)
0808 06:08:49.493798 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:49.493802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:49.493942 (+ 140us) server_negotiation.cc:407] Connection header received
0808 06:08:49.494079 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:49.494102 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:49.494132 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:49.494217 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:49.494852 (+ 635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:49.494855 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:49.494856 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:08:49.494866 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:49.494889 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:08:49.494899 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:49.494958 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:49.495051 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 06:08:49.495100 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 06:08:49.495427 1825 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7df1629fa3634306b5eeca2c74239018 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7df1629fa3634306b5eeca2c74239018" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38099 } }
I20260808 06:08:49.495703 1825 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7df1629fa3634306b5eeca2c74239018 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:08:49.495774 1825 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7df1629fa3634306b5eeca2c74239018 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7df1629fa3634306b5eeca2c74239018, State: Initialized, Role: FOLLOWER
I20260808 06:08:49.496196 1825 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7df1629fa3634306b5eeca2c74239018 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7df1629fa3634306b5eeca2c74239018" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38099 } }
I20260808 06:08:49.496322 1825 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7df1629fa3634306b5eeca2c74239018 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:08:49.496367 1825 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7df1629fa3634306b5eeca2c74239018 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:08:49.496434 1825 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7df1629fa3634306b5eeca2c74239018 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:08:49.497254 1825 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7df1629fa3634306b5eeca2c74239018 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7df1629fa3634306b5eeca2c74239018" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38099 } }
I20260808 06:08:49.497541 1825 leader_election.cc:302] T 00000000000000000000000000000000 P 7df1629fa3634306b5eeca2c74239018 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7df1629fa3634306b5eeca2c74239018; no voters:
I20260808 06:08:49.497855 1825 leader_election.cc:288] T 00000000000000000000000000000000 P 7df1629fa3634306b5eeca2c74239018 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:08:49.497980 1841 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7df1629fa3634306b5eeca2c74239018 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:08:49.498230 1841 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7df1629fa3634306b5eeca2c74239018 [term 1 LEADER]: Becoming Leader. State: Replica: 7df1629fa3634306b5eeca2c74239018, State: Running, Role: LEADER
I20260808 06:08:49.498512 1841 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7df1629fa3634306b5eeca2c74239018 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7df1629fa3634306b5eeca2c74239018" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38099 } }
I20260808 06:08:49.499188 1825 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7df1629fa3634306b5eeca2c74239018 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:08:49.499781 1841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7df1629fa3634306b5eeca2c74239018 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7df1629fa3634306b5eeca2c74239018. Latest consensus state: current_term: 1 leader_uuid: "7df1629fa3634306b5eeca2c74239018" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7df1629fa3634306b5eeca2c74239018" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38099 } } }
I20260808 06:08:49.500492 1841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7df1629fa3634306b5eeca2c74239018 [sys.catalog]: This master's current role is: LEADER
I20260808 06:08:49.500376 1843 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7df1629fa3634306b5eeca2c74239018 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7df1629fa3634306b5eeca2c74239018" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7df1629fa3634306b5eeca2c74239018" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38099 } } }
I20260808 06:08:49.500838 1843 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7df1629fa3634306b5eeca2c74239018 [sys.catalog]: This master's current role is: LEADER
I20260808 06:08:49.501763 1855 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:08:49.505061 1855 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:08:49.511891 1855 catalog_manager.cc:1347] Generated new cluster ID: 3f37cde148564db2ba5aaafc96b66d43
I20260808 06:08:49.511955 1855 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:08:49.527753 1855 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:08:49.528062 1855 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:08:49.535105 1855 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7df1629fa3634306b5eeca2c74239018: Generated new TSK 0
I20260808 06:08:49.535836 1855 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 06:08:49.540369 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 1723
2026-08-08T06:08:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1487 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:08:49 dist-test-slave-zj8q krb5kdc[1873](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:08:49 dist-test-slave-zj8q krb5kdc[1873](info): set up 2 sockets
Aug 08 06:08:49 dist-test-slave-zj8q krb5kdc[1873](info): commencing operation
krb5kdc: starting...
W20260808 06:08:50.688117 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:08:50 dist-test-slave-zj8q krb5kdc[1873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:08:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:08:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 06:08:50.808346 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41001
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41001
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:08:50.916908 1891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:50.917140 1891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:50.917178 1891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:50.921036 1891 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:08:50.921087 1891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:50.921104 1891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:50.921119 1891 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:08:50.921132 1891 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:08:50.926098 1891 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41001
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:50.927119 1891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:50.928117 1891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:08:50.936440 1898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:50.936466 1903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:50.936466 1897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:50.936645 1891 server_base.cc:1034] running on GCE node
I20260808 06:08:50.937268 1891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:50.938319 1891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:50.939476 1891 hybrid_clock.cc:648] HybridClock initialized: now 1786169330939445 us; error 41 us; skew 500 ppm
Aug 08 06:08:50 dist-test-slave-zj8q krb5kdc[1873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169330, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:08:50.945930 1891 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 06:08:50.946825 1891 webserver.cc:460] Webserver started at http://127.0.0.1:33717/ using document root <none> and password file <none>
I20260808 06:08:50.947358 1891 fs_manager.cc:359] Metadata directory not provided
I20260808 06:08:50.947419 1891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:08:50.947676 1891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:08:50.949832 1891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "3d53c7eb3d7149fa902054702bb2b123"
format_stamp: "Formatted at 2026-08-08 06:08:50 on dist-test-slave-zj8q"
server_key: "46a41b9a686c7e7da3a6b02762dd10a1"
server_key_iv: "124fa9dcb62d2bae76e6c3e79ca2d295"
server_key_version: "encryptionkey@0"
I20260808 06:08:50.950410 1891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "3d53c7eb3d7149fa902054702bb2b123"
format_stamp: "Formatted at 2026-08-08 06:08:50 on dist-test-slave-zj8q"
server_key: "46a41b9a686c7e7da3a6b02762dd10a1"
server_key_iv: "124fa9dcb62d2bae76e6c3e79ca2d295"
server_key_version: "encryptionkey@0"
I20260808 06:08:50.953985 1891 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 06:08:50.956385 1910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:50.957510 1891 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:08:50.957631 1891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "3d53c7eb3d7149fa902054702bb2b123"
format_stamp: "Formatted at 2026-08-08 06:08:50 on dist-test-slave-zj8q"
server_key: "46a41b9a686c7e7da3a6b02762dd10a1"
server_key_iv: "124fa9dcb62d2bae76e6c3e79ca2d295"
server_key_version: "encryptionkey@0"
I20260808 06:08:50.957724 1891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:08:50.974228 1891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:08:50.976910 1891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:08:50.977080 1891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:08:50.984133 1891 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41001
I20260808 06:08:50.984249 1984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41001 every 8 connection(s)
I20260808 06:08:50.985081 1891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:08:50.987926 1985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:08:50.993595 1985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d53c7eb3d7149fa902054702bb2b123: Bootstrap starting.
I20260808 06:08:50.994572 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 1891
I20260808 06:08:50.994724 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:08:50.995044 1630 external_mini_cluster.cc:1428] Setting key 6c8e31b042465457898c9a0d48f73a8b
I20260808 06:08:50.996567 1985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d53c7eb3d7149fa902054702bb2b123: Neither blocks nor log segments found. Creating new log.
I20260808 06:08:50.997267 1985 log.cc:824] T 00000000000000000000000000000000 P 3d53c7eb3d7149fa902054702bb2b123: Log is configured to *not* fsync() on all Append() calls
I20260808 06:08:50.999648 1985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d53c7eb3d7149fa902054702bb2b123: No bootstrap required, opened a new log
Aug 08 06:08:51 dist-test-slave-zj8q krb5kdc[1873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 06:08:51.003899 1985 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d53c7eb3d7149fa902054702bb2b123 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d53c7eb3d7149fa902054702bb2b123" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41001 } }
I20260808 06:08:51.004181 1985 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d53c7eb3d7149fa902054702bb2b123 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:08:51.004241 1985 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d53c7eb3d7149fa902054702bb2b123 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d53c7eb3d7149fa902054702bb2b123, State: Initialized, Role: FOLLOWER
I20260808 06:08:51.004674 1985 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d53c7eb3d7149fa902054702bb2b123 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d53c7eb3d7149fa902054702bb2b123" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41001 } }
I20260808 06:08:51.004807 1985 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d53c7eb3d7149fa902054702bb2b123 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:08:51.004873 1985 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d53c7eb3d7149fa902054702bb2b123 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:08:51.004974 1985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d53c7eb3d7149fa902054702bb2b123 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:08:51.005821 1985 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d53c7eb3d7149fa902054702bb2b123 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d53c7eb3d7149fa902054702bb2b123" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41001 } }
I20260808 06:08:51.006165 1985 leader_election.cc:302] T 00000000000000000000000000000000 P 3d53c7eb3d7149fa902054702bb2b123 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d53c7eb3d7149fa902054702bb2b123; no voters:
I20260808 06:08:51.006491 1985 leader_election.cc:288] T 00000000000000000000000000000000 P 3d53c7eb3d7149fa902054702bb2b123 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:08:51.006570 1994 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d53c7eb3d7149fa902054702bb2b123 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:08:51.006795 1994 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d53c7eb3d7149fa902054702bb2b123 [term 1 LEADER]: Becoming Leader. State: Replica: 3d53c7eb3d7149fa902054702bb2b123, State: Running, Role: LEADER
I20260808 06:08:51.007177 1994 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d53c7eb3d7149fa902054702bb2b123 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d53c7eb3d7149fa902054702bb2b123" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41001 } }
I20260808 06:08:51.007711 1985 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d53c7eb3d7149fa902054702bb2b123 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:08:51.008849 1995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d53c7eb3d7149fa902054702bb2b123 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d53c7eb3d7149fa902054702bb2b123" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d53c7eb3d7149fa902054702bb2b123" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41001 } } }
I20260808 06:08:51.008960 1995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d53c7eb3d7149fa902054702bb2b123 [sys.catalog]: This master's current role is: LEADER
I20260808 06:08:51.009326 1998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d53c7eb3d7149fa902054702bb2b123 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d53c7eb3d7149fa902054702bb2b123. Latest consensus state: current_term: 1 leader_uuid: "3d53c7eb3d7149fa902054702bb2b123" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d53c7eb3d7149fa902054702bb2b123" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41001 } } }
I20260808 06:08:51.009390 2008 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:08:51.009449 1998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d53c7eb3d7149fa902054702bb2b123 [sys.catalog]: This master's current role is: LEADER
I20260808 06:08:51.009756 1990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:50.996144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39471 (local address 127.0.0.1:41001)
0808 06:08:50.996535 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:50.996550 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:50.996586 (+ 36us) server_negotiation.cc:407] Connection header received
0808 06:08:50.997239 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:50.997256 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:50.997636 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:50.998004 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:50.998917 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:50.999921 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:51.001050 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:51.001502 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:51.001621 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:51.004431 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:51.004436 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:51.004450 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:08:51.004483 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:51.006253 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:51.006760 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:51.006767 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:51.006773 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:51.006937 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:51.007241 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:51.007244 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:51.007246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:51.007749 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:51.007969 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:51.008406 (+ 437us) server_negotiation.cc:299] Negotiation successful
0808 06:08:51.008733 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":105,"threads_started":1}
I20260808 06:08:51.012730 2008 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:08:51.013253 1630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 06:08:51.018079 2008 catalog_manager.cc:1347] Generated new cluster ID: 3f00352adf3c4c8783f4c57eb7711d83
I20260808 06:08:51.018137 2008 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:08:51.023605 1990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:51.016524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48757 (local address 127.0.0.1:41001)
0808 06:08:51.016619 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:51.016623 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:51.016919 (+ 296us) server_negotiation.cc:407] Connection header received
0808 06:08:51.017026 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:51.017029 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:51.017071 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:51.017145 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:51.017942 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:51.018563 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:51.019352 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:51.019590 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:51.019682 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:51.020881 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:51.020885 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:51.020887 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:08:51.020911 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:51.022139 (+ 1228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:51.022550 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:51.022554 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:51.022556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:51.022633 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:51.022865 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:51.022868 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:51.022871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:51.023035 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:51.023170 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:51.023364 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 06:08:51.023464 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 06:08:51.030620 2008 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:08:51.031848 2008 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:08:51.050490 2008 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d53c7eb3d7149fa902054702bb2b123: Generated new TSK 0
I20260808 06:08:51.051337 2008 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 06:08:51.059424 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 1891
2026-08-08T06:08:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1525 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:08:51 dist-test-slave-zj8q krb5kdc[2030](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:08:51 dist-test-slave-zj8q krb5kdc[2030](info): set up 2 sockets
Aug 08 06:08:51 dist-test-slave-zj8q krb5kdc[2030](info): commencing operation
krb5kdc: starting...
W20260808 06:08:52.207657 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:08:52 dist-test-slave-zj8q krb5kdc[2030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:08:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:08:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 06:08:52.337527 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35727
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35727
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:08:52.447103 2046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:52.447366 2046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:52.447422 2046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:52.451653 2046 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:08:52.451710 2046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:52.451736 2046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:52.451757 2046 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:08:52.451786 2046 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:08:52.456907 2046 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35727
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:52.457866 2046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:52.458864 2046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:08:52.467186 2061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:52.467231 2046 server_base.cc:1034] running on GCE node
W20260808 06:08:52.467226 2057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:52.467180 2055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:52.467908 2046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:52.468979 2046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:52.470135 2046 hybrid_clock.cc:648] HybridClock initialized: now 1786169332470121 us; error 26 us; skew 500 ppm
Aug 08 06:08:52 dist-test-slave-zj8q krb5kdc[2030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169332, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:08:52.472870 2046 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 06:08:52.473732 2046 webserver.cc:460] Webserver started at http://127.0.0.1:43617/ using document root <none> and password file <none>
I20260808 06:08:52.474308 2046 fs_manager.cc:359] Metadata directory not provided
I20260808 06:08:52.474377 2046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:08:52.474584 2046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:08:52.476529 2046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "aee0635a4fd24c598cd0872d01e1a6bc"
format_stamp: "Formatted at 2026-08-08 06:08:52 on dist-test-slave-zj8q"
server_key: "faa9cbaa7400050d76845a50d4116e51"
server_key_iv: "709204d61359044a3716d786637e91c1"
server_key_version: "encryptionkey@0"
I20260808 06:08:52.477072 2046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "aee0635a4fd24c598cd0872d01e1a6bc"
format_stamp: "Formatted at 2026-08-08 06:08:52 on dist-test-slave-zj8q"
server_key: "faa9cbaa7400050d76845a50d4116e51"
server_key_iv: "709204d61359044a3716d786637e91c1"
server_key_version: "encryptionkey@0"
I20260808 06:08:52.480676 2046 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 06:08:52.483026 2071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:52.484205 2046 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:08:52.484319 2046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "aee0635a4fd24c598cd0872d01e1a6bc"
format_stamp: "Formatted at 2026-08-08 06:08:52 on dist-test-slave-zj8q"
server_key: "faa9cbaa7400050d76845a50d4116e51"
server_key_iv: "709204d61359044a3716d786637e91c1"
server_key_version: "encryptionkey@0"
I20260808 06:08:52.484412 2046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:08:52.495559 2046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:08:52.498473 2046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:08:52.498646 2046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:08:52.505755 2046 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35727
I20260808 06:08:52.505787 2154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35727 every 8 connection(s)
I20260808 06:08:52.506690 2046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:08:52.509413 2155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:08:52.513960 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 2046
I20260808 06:08:52.514092 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:08:52.514379 1630 external_mini_cluster.cc:1428] Setting key d083e1805e2a2f275cae707afe3b447b
I20260808 06:08:52.514830 2155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aee0635a4fd24c598cd0872d01e1a6bc: Bootstrap starting.
I20260808 06:08:52.517129 2155 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aee0635a4fd24c598cd0872d01e1a6bc: Neither blocks nor log segments found. Creating new log.
I20260808 06:08:52.517948 2155 log.cc:824] T 00000000000000000000000000000000 P aee0635a4fd24c598cd0872d01e1a6bc: Log is configured to *not* fsync() on all Append() calls
I20260808 06:08:52.520031 2155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aee0635a4fd24c598cd0872d01e1a6bc: No bootstrap required, opened a new log
I20260808 06:08:52.522776 2160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:52.515602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52209 (local address 127.0.0.1:35727)
0808 06:08:52.515982 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:52.515999 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:52.516033 (+ 34us) server_negotiation.cc:407] Connection header received
0808 06:08:52.516632 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:52.516653 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:52.516973 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:52.517264 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:52.517587 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:52.518631 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:52.519795 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:52.520283 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:52.520397 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:52.521178 (+ 781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:52.521184 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:52.521197 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:08:52.521552 (+ 355us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:52.521602 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:08:52.521624 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:52.521700 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:52.521845 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 06:08:52.522031 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":103,"threads_started":1}
I20260808 06:08:52.523844 2155 raft_consensus.cc:348] T 00000000000000000000000000000000 P aee0635a4fd24c598cd0872d01e1a6bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aee0635a4fd24c598cd0872d01e1a6bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35727 } }
I20260808 06:08:52.524057 2155 raft_consensus.cc:374] T 00000000000000000000000000000000 P aee0635a4fd24c598cd0872d01e1a6bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:08:52.524096 2155 raft_consensus.cc:729] T 00000000000000000000000000000000 P aee0635a4fd24c598cd0872d01e1a6bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aee0635a4fd24c598cd0872d01e1a6bc, State: Initialized, Role: FOLLOWER
I20260808 06:08:52.524535 2155 consensus_queue.cc:260] T 00000000000000000000000000000000 P aee0635a4fd24c598cd0872d01e1a6bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aee0635a4fd24c598cd0872d01e1a6bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35727 } }
I20260808 06:08:52.524715 2155 raft_consensus.cc:388] T 00000000000000000000000000000000 P aee0635a4fd24c598cd0872d01e1a6bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:08:52.524782 2155 raft_consensus.cc:482] T 00000000000000000000000000000000 P aee0635a4fd24c598cd0872d01e1a6bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:08:52.524880 2155 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aee0635a4fd24c598cd0872d01e1a6bc [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:08:52.525478 1630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 06:08:52.525864 2155 raft_consensus.cc:504] T 00000000000000000000000000000000 P aee0635a4fd24c598cd0872d01e1a6bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aee0635a4fd24c598cd0872d01e1a6bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35727 } }
I20260808 06:08:52.526198 2155 leader_election.cc:302] T 00000000000000000000000000000000 P aee0635a4fd24c598cd0872d01e1a6bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aee0635a4fd24c598cd0872d01e1a6bc; no voters:
I20260808 06:08:52.526508 2155 leader_election.cc:288] T 00000000000000000000000000000000 P aee0635a4fd24c598cd0872d01e1a6bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:08:52.526611 2163 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aee0635a4fd24c598cd0872d01e1a6bc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:08:52.526887 2163 raft_consensus.cc:686] T 00000000000000000000000000000000 P aee0635a4fd24c598cd0872d01e1a6bc [term 1 LEADER]: Becoming Leader. State: Replica: aee0635a4fd24c598cd0872d01e1a6bc, State: Running, Role: LEADER
I20260808 06:08:52.527200 2163 consensus_queue.cc:237] T 00000000000000000000000000000000 P aee0635a4fd24c598cd0872d01e1a6bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aee0635a4fd24c598cd0872d01e1a6bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35727 } }
I20260808 06:08:52.527725 2155 sys_catalog.cc:564] T 00000000000000000000000000000000 P aee0635a4fd24c598cd0872d01e1a6bc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:08:52.528420 2163 sys_catalog.cc:455] T 00000000000000000000000000000000 P aee0635a4fd24c598cd0872d01e1a6bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader aee0635a4fd24c598cd0872d01e1a6bc. Latest consensus state: current_term: 1 leader_uuid: "aee0635a4fd24c598cd0872d01e1a6bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aee0635a4fd24c598cd0872d01e1a6bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35727 } } }
I20260808 06:08:52.528560 2163 sys_catalog.cc:458] T 00000000000000000000000000000000 P aee0635a4fd24c598cd0872d01e1a6bc [sys.catalog]: This master's current role is: LEADER
I20260808 06:08:52.529117 2169 sys_catalog.cc:455] T 00000000000000000000000000000000 P aee0635a4fd24c598cd0872d01e1a6bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aee0635a4fd24c598cd0872d01e1a6bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aee0635a4fd24c598cd0872d01e1a6bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35727 } } }
I20260808 06:08:52.529227 2169 sys_catalog.cc:458] T 00000000000000000000000000000000 P aee0635a4fd24c598cd0872d01e1a6bc [sys.catalog]: This master's current role is: LEADER
I20260808 06:08:52.529644 2186 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:08:52.532397 2160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:52.527535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42075 (local address 127.0.0.1:35727)
0808 06:08:52.527628 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:52.527632 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:52.528107 (+ 475us) server_negotiation.cc:407] Connection header received
0808 06:08:52.528206 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:52.528209 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:52.528254 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:52.528326 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:52.528689 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:52.529531 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:52.530633 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:52.530937 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:52.531033 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:52.531804 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:52.531807 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:52.531810 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:08:52.531822 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:52.531850 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:08:52.531864 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:52.531927 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:52.532208 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 06:08:52.532259 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":27}
I20260808 06:08:52.533044 2186 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:08:52.538913 2186 catalog_manager.cc:1347] Generated new cluster ID: 7372e1bac4114cdca7f7a47c1be72dde
I20260808 06:08:52.538967 2186 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:08:52.552163 2186 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:08:52.553454 2186 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:08:52.560577 2186 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aee0635a4fd24c598cd0872d01e1a6bc: Generated new TSK 0
I20260808 06:08:52.561400 2186 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 06:08:52.572963 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 2046
2026-08-08T06:08:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1514 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:08:52 dist-test-slave-zj8q krb5kdc[2206](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:08:52 dist-test-slave-zj8q krb5kdc[2206](info): set up 2 sockets
Aug 08 06:08:52 dist-test-slave-zj8q krb5kdc[2206](info): commencing operation
krb5kdc: starting...
W20260808 06:08:53.717458 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:08:53 dist-test-slave-zj8q krb5kdc[2206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:08:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:08:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 06:08:53.833808 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38015
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38015
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:08:53.941543 2233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:53.941805 2233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:53.941864 2233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:53.945782 2233 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:08:53.945839 2233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:53.945865 2233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:53.945885 2233 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:08:53.945906 2233 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:08:53.951076 2233 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38015
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:53.952065 2233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:53.953094 2233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:08:53.961360 2243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:53.961426 2233 server_base.cc:1034] running on GCE node
W20260808 06:08:53.961323 2242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:53.961330 2245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:53.962136 2233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:53.963189 2233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:53.964349 2233 hybrid_clock.cc:648] HybridClock initialized: now 1786169333964333 us; error 27 us; skew 500 ppm
Aug 08 06:08:53 dist-test-slave-zj8q krb5kdc[2206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169333, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:08:53.967012 2233 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 06:08:53.967813 2233 webserver.cc:460] Webserver started at http://127.0.0.1:41349/ using document root <none> and password file <none>
I20260808 06:08:53.968339 2233 fs_manager.cc:359] Metadata directory not provided
I20260808 06:08:53.968413 2233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:08:53.968609 2233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:08:53.970568 2233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "b7853a5fe0c4421ebfaba6c424a30496"
format_stamp: "Formatted at 2026-08-08 06:08:53 on dist-test-slave-zj8q"
server_key: "602bf272cba9f737d0ae06dde6fc2e27"
server_key_iv: "32b89489ea6102088d6b37a892c63188"
server_key_version: "encryptionkey@0"
I20260808 06:08:53.971056 2233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "b7853a5fe0c4421ebfaba6c424a30496"
format_stamp: "Formatted at 2026-08-08 06:08:53 on dist-test-slave-zj8q"
server_key: "602bf272cba9f737d0ae06dde6fc2e27"
server_key_iv: "32b89489ea6102088d6b37a892c63188"
server_key_version: "encryptionkey@0"
I20260808 06:08:53.974483 2233 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 06:08:53.976696 2265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:53.977797 2233 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:08:53.977897 2233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "b7853a5fe0c4421ebfaba6c424a30496"
format_stamp: "Formatted at 2026-08-08 06:08:53 on dist-test-slave-zj8q"
server_key: "602bf272cba9f737d0ae06dde6fc2e27"
server_key_iv: "32b89489ea6102088d6b37a892c63188"
server_key_version: "encryptionkey@0"
I20260808 06:08:53.977979 2233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:08:53.982286 2233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:08:53.984867 2233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:08:53.985006 2233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:08:53.991703 2233 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38015
I20260808 06:08:53.991730 2348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38015 every 8 connection(s)
I20260808 06:08:53.992570 2233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:08:53.995342 2352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:08:53.999135 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 2233
I20260808 06:08:53.999257 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:08:53.999505 1630 external_mini_cluster.cc:1428] Setting key 4a01d858e183dd1dfa842cf7ccd6040d
I20260808 06:08:54.001044 2352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7853a5fe0c4421ebfaba6c424a30496: Bootstrap starting.
I20260808 06:08:54.003811 2352 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7853a5fe0c4421ebfaba6c424a30496: Neither blocks nor log segments found. Creating new log.
I20260808 06:08:54.004717 2352 log.cc:824] T 00000000000000000000000000000000 P b7853a5fe0c4421ebfaba6c424a30496: Log is configured to *not* fsync() on all Append() calls
I20260808 06:08:54.005096 2357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:54.001005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59123 (local address 127.0.0.1:38015)
0808 06:08:54.001447 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:54.001466 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:54.001507 (+ 41us) server_negotiation.cc:407] Connection header received
0808 06:08:54.002371 (+ 864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:54.002395 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:54.002747 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:54.003093 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:54.003724 (+ 631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:54.003729 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:54.003743 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:08:54.003755 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:54.003816 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:08:54.003845 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:54.003948 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:54.004056 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 06:08:54.004220 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":123,"threads_started":1}
I20260808 06:08:54.006695 2352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7853a5fe0c4421ebfaba6c424a30496: No bootstrap required, opened a new log
I20260808 06:08:54.007833 1630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 06:08:54.011044 2352 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7853a5fe0c4421ebfaba6c424a30496 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7853a5fe0c4421ebfaba6c424a30496" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38015 } }
I20260808 06:08:54.011329 2352 raft_consensus.cc:374] T 00000000000000000000000000000000 P b7853a5fe0c4421ebfaba6c424a30496 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:08:54.011401 2352 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7853a5fe0c4421ebfaba6c424a30496 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7853a5fe0c4421ebfaba6c424a30496, State: Initialized, Role: FOLLOWER
I20260808 06:08:54.011723 2357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:54.010101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33657 (local address 127.0.0.1:38015)
0808 06:08:54.010243 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:54.010247 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:54.010429 (+ 182us) server_negotiation.cc:407] Connection header received
0808 06:08:54.010555 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:54.010557 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:54.010584 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:54.010644 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:54.011328 (+ 684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:54.011331 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:54.011334 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:08:54.011347 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:54.011374 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:08:54.011386 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:54.011465 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:54.011543 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 06:08:54.011575 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 06:08:54.011870 2352 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7853a5fe0c4421ebfaba6c424a30496 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7853a5fe0c4421ebfaba6c424a30496" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38015 } }
I20260808 06:08:54.012027 2352 raft_consensus.cc:388] T 00000000000000000000000000000000 P b7853a5fe0c4421ebfaba6c424a30496 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:08:54.012092 2352 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7853a5fe0c4421ebfaba6c424a30496 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:08:54.012188 2352 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7853a5fe0c4421ebfaba6c424a30496 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:08:54.013208 2352 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7853a5fe0c4421ebfaba6c424a30496 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7853a5fe0c4421ebfaba6c424a30496" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38015 } }
I20260808 06:08:54.013558 2352 leader_election.cc:302] T 00000000000000000000000000000000 P b7853a5fe0c4421ebfaba6c424a30496 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7853a5fe0c4421ebfaba6c424a30496; no voters:
I20260808 06:08:54.013849 2352 leader_election.cc:288] T 00000000000000000000000000000000 P b7853a5fe0c4421ebfaba6c424a30496 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:08:54.013964 2370 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7853a5fe0c4421ebfaba6c424a30496 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:08:54.014200 2370 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7853a5fe0c4421ebfaba6c424a30496 [term 1 LEADER]: Becoming Leader. State: Replica: b7853a5fe0c4421ebfaba6c424a30496, State: Running, Role: LEADER
I20260808 06:08:54.014565 2370 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7853a5fe0c4421ebfaba6c424a30496 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7853a5fe0c4421ebfaba6c424a30496" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38015 } }
I20260808 06:08:54.015126 2352 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7853a5fe0c4421ebfaba6c424a30496 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:08:54.016748 2371 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7853a5fe0c4421ebfaba6c424a30496 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7853a5fe0c4421ebfaba6c424a30496. Latest consensus state: current_term: 1 leader_uuid: "b7853a5fe0c4421ebfaba6c424a30496" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7853a5fe0c4421ebfaba6c424a30496" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38015 } } }
I20260808 06:08:54.016878 2371 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7853a5fe0c4421ebfaba6c424a30496 [sys.catalog]: This master's current role is: LEADER
I20260808 06:08:54.017035 2373 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7853a5fe0c4421ebfaba6c424a30496 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7853a5fe0c4421ebfaba6c424a30496" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7853a5fe0c4421ebfaba6c424a30496" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38015 } } }
I20260808 06:08:54.017122 2373 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7853a5fe0c4421ebfaba6c424a30496 [sys.catalog]: This master's current role is: LEADER
I20260808 06:08:54.018332 2382 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:08:54.021399 2382 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:08:54.026916 2382 catalog_manager.cc:1347] Generated new cluster ID: 6e7ff61586a2476eaede25263a6ace49
I20260808 06:08:54.026981 2382 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:08:54.042753 2382 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:08:54.043073 2382 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:08:54.048231 2382 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7853a5fe0c4421ebfaba6c424a30496: Generated new TSK 0
I20260808 06:08:54.048830 2382 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 06:08:54.052706 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 2233
2026-08-08T06:08:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1475 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:08:54 dist-test-slave-zj8q krb5kdc[2405](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:08:54 dist-test-slave-zj8q krb5kdc[2405](info): set up 2 sockets
Aug 08 06:08:54 dist-test-slave-zj8q krb5kdc[2405](info): commencing operation
krb5kdc: starting...
W20260808 06:08:55.198359 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:08:55 dist-test-slave-zj8q krb5kdc[2405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169335, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:08:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:08:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 06:08:55.318629 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36889
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:08:55.427380 2421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:55.427630 2421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:55.427690 2421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:55.431560 2421 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:08:55.431632 2421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:55.431658 2421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:55.431679 2421 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:08:55.431708 2421 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:08:55.436856 2421 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36889
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:55.437825 2421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:55.438836 2421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 06:08:55.446930 2421 server_base.cc:1034] running on GCE node
W20260808 06:08:55.446874 2428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:55.446874 2433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:55.446913 2430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:55.447633 2421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:55.448661 2421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:55.449827 2421 hybrid_clock.cc:648] HybridClock initialized: now 1786169335449804 us; error 37 us; skew 500 ppm
Aug 08 06:08:55 dist-test-slave-zj8q krb5kdc[2405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169335, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:08:55.452469 2421 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 06:08:55.453294 2421 webserver.cc:460] Webserver started at http://127.0.0.1:36401/ using document root <none> and password file <none>
I20260808 06:08:55.453822 2421 fs_manager.cc:359] Metadata directory not provided
I20260808 06:08:55.453889 2421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:08:55.454071 2421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:08:55.456092 2421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "77ed9b741e814d46a6ed02864b22a988"
format_stamp: "Formatted at 2026-08-08 06:08:55 on dist-test-slave-zj8q"
server_key: "98a2392d6347945d5952705c1b3e1544"
server_key_iv: "bddc399509f6206f031043bbee5162d1"
server_key_version: "encryptionkey@0"
I20260808 06:08:55.456660 2421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "77ed9b741e814d46a6ed02864b22a988"
format_stamp: "Formatted at 2026-08-08 06:08:55 on dist-test-slave-zj8q"
server_key: "98a2392d6347945d5952705c1b3e1544"
server_key_iv: "bddc399509f6206f031043bbee5162d1"
server_key_version: "encryptionkey@0"
I20260808 06:08:55.460210 2421 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 06:08:55.462448 2443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:55.463593 2421 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 06:08:55.463717 2421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "77ed9b741e814d46a6ed02864b22a988"
format_stamp: "Formatted at 2026-08-08 06:08:55 on dist-test-slave-zj8q"
server_key: "98a2392d6347945d5952705c1b3e1544"
server_key_iv: "bddc399509f6206f031043bbee5162d1"
server_key_version: "encryptionkey@0"
I20260808 06:08:55.463809 2421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:08:55.486630 2421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:08:55.489234 2421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:08:55.489405 2421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:08:55.496299 2421 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36889
I20260808 06:08:55.496327 2536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36889 every 8 connection(s)
I20260808 06:08:55.497206 2421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:08:55.499933 2537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:08:55.504792 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 2421
I20260808 06:08:55.504936 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:08:55.505228 1630 external_mini_cluster.cc:1428] Setting key b2881307496dbe7773785a7631143f6e
I20260808 06:08:55.505568 2537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77ed9b741e814d46a6ed02864b22a988: Bootstrap starting.
I20260808 06:08:55.508011 2537 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77ed9b741e814d46a6ed02864b22a988: Neither blocks nor log segments found. Creating new log.
I20260808 06:08:55.508901 2537 log.cc:824] T 00000000000000000000000000000000 P 77ed9b741e814d46a6ed02864b22a988: Log is configured to *not* fsync() on all Append() calls
I20260808 06:08:55.511112 2537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77ed9b741e814d46a6ed02864b22a988: No bootstrap required, opened a new log
Aug 08 06:08:55 dist-test-slave-zj8q krb5kdc[2405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169335, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 06:08:55.515300 2537 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77ed9b741e814d46a6ed02864b22a988 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77ed9b741e814d46a6ed02864b22a988" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36889 } }
I20260808 06:08:55.515581 2537 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77ed9b741e814d46a6ed02864b22a988 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:08:55.515645 2537 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77ed9b741e814d46a6ed02864b22a988 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77ed9b741e814d46a6ed02864b22a988, State: Initialized, Role: FOLLOWER
I20260808 06:08:55.516136 2537 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77ed9b741e814d46a6ed02864b22a988 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77ed9b741e814d46a6ed02864b22a988" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36889 } }
I20260808 06:08:55.516296 2537 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77ed9b741e814d46a6ed02864b22a988 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:08:55.516359 2537 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77ed9b741e814d46a6ed02864b22a988 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:08:55.516451 2537 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77ed9b741e814d46a6ed02864b22a988 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:08:55.517720 2537 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77ed9b741e814d46a6ed02864b22a988 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77ed9b741e814d46a6ed02864b22a988" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36889 } }
I20260808 06:08:55.518093 2537 leader_election.cc:302] T 00000000000000000000000000000000 P 77ed9b741e814d46a6ed02864b22a988 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77ed9b741e814d46a6ed02864b22a988; no voters:
I20260808 06:08:55.518437 2537 leader_election.cc:288] T 00000000000000000000000000000000 P 77ed9b741e814d46a6ed02864b22a988 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:08:55.518532 2548 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77ed9b741e814d46a6ed02864b22a988 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:08:55.518759 2548 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77ed9b741e814d46a6ed02864b22a988 [term 1 LEADER]: Becoming Leader. State: Replica: 77ed9b741e814d46a6ed02864b22a988, State: Running, Role: LEADER
I20260808 06:08:55.519129 2548 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77ed9b741e814d46a6ed02864b22a988 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77ed9b741e814d46a6ed02864b22a988" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36889 } }
I20260808 06:08:55.519797 2537 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77ed9b741e814d46a6ed02864b22a988 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:08:55.520030 2542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:55.506580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.236:39359 (local address 127.0.0.1:36889)
0808 06:08:55.506969 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:55.506984 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:55.507018 (+ 34us) server_negotiation.cc:407] Connection header received
0808 06:08:55.507651 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:55.507668 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:55.507992 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:55.508288 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:55.509264 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:55.510323 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:55.511474 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:55.511970 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:55.512084 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:55.514839 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:55.514846 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:55.514861 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:08:55.514892 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:55.516794 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:55.517314 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:55.517321 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:55.517326 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:55.517476 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:55.517776 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:55.517780 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:55.517782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:55.518334 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:55.518549 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:55.518905 (+ 356us) server_negotiation.cc:299] Negotiation successful
0808 06:08:55.519131 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":255,"thread_start_us":108,"threads_started":1}
I20260808 06:08:55.520210 2548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77ed9b741e814d46a6ed02864b22a988 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77ed9b741e814d46a6ed02864b22a988. Latest consensus state: current_term: 1 leader_uuid: "77ed9b741e814d46a6ed02864b22a988" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77ed9b741e814d46a6ed02864b22a988" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36889 } } }
I20260808 06:08:55.520367 2548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77ed9b741e814d46a6ed02864b22a988 [sys.catalog]: This master's current role is: LEADER
I20260808 06:08:55.520617 2548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77ed9b741e814d46a6ed02864b22a988 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77ed9b741e814d46a6ed02864b22a988" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77ed9b741e814d46a6ed02864b22a988" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36889 } } }
I20260808 06:08:55.520756 2548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77ed9b741e814d46a6ed02864b22a988 [sys.catalog]: This master's current role is: LEADER
I20260808 06:08:55.521065 2555 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:08:55.523025 1630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 06:08:55.524593 2555 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:08:55.530326 2555 catalog_manager.cc:1347] Generated new cluster ID: b8ac732971b341a6a273a8e1233ee4a5
I20260808 06:08:55.530431 2555 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:08:55.533525 2542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:55.525173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.236:33671 (local address 127.0.0.1:36889)
0808 06:08:55.525277 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:55.525281 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:55.525601 (+ 320us) server_negotiation.cc:407] Connection header received
0808 06:08:55.525698 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:55.525701 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:55.525743 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:55.525804 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:55.526482 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:55.527173 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:55.528348 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:55.528666 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:55.528756 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:55.530588 (+ 1832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:55.530592 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:55.530595 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:08:55.530616 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:55.531920 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:55.532361 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:55.532365 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:55.532367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:55.532439 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:08:55.532740 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:08:55.532744 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:08:55.532746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:08:55.532913 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:55.533043 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:55.533305 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 06:08:55.533390 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 06:08:55.553759 2555 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:08:55.555032 2555 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:08:55.567577 2555 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77ed9b741e814d46a6ed02864b22a988: Generated new TSK 0
I20260808 06:08:55.568436 2555 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 06:08:55.574821 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 2421
W20260808 06:08:55.591564 2398 connection.cc:535] client connection to 127.0.0.1:36889 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36889): Connection reset by peer (error 104)
2026-08-08T06:08:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1522 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:08:55 dist-test-slave-zj8q krb5kdc[2583](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:08:55 dist-test-slave-zj8q krb5kdc[2583](info): set up 2 sockets
Aug 08 06:08:55 dist-test-slave-zj8q krb5kdc[2583](info): commencing operation
krb5kdc: starting...
W20260808 06:08:56.724351 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:08:56 dist-test-slave-zj8q krb5kdc[2583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:08:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:08:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 06:08:56.842357 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41799
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:08:56.950881 2605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:56.951145 2605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:56.951206 2605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:56.955188 2605 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:08:56.955245 2605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:56.955269 2605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:56.955292 2605 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:08:56.955322 2605 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:08:56.960350 2605 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41799
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:56.961304 2605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:56.962292 2605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:08:56.970449 2620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:56.970427 2617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:56.970423 2615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:56.970654 2605 server_base.cc:1034] running on GCE node
I20260808 06:08:56.971256 2605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:56.972281 2605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:56.973444 2605 hybrid_clock.cc:648] HybridClock initialized: now 1786169336973428 us; error 26 us; skew 500 ppm
Aug 08 06:08:56 dist-test-slave-zj8q krb5kdc[2583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169336, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:08:56.981328 2605 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 06:08:56.982205 2605 webserver.cc:460] Webserver started at http://127.0.0.1:44053/ using document root <none> and password file <none>
I20260808 06:08:56.982741 2605 fs_manager.cc:359] Metadata directory not provided
I20260808 06:08:56.982805 2605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:08:56.983002 2605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:08:56.984978 2605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "198ef032fd9048e09d4d2436b0954f55"
format_stamp: "Formatted at 2026-08-08 06:08:56 on dist-test-slave-zj8q"
server_key: "3fbe0ac2a1b644a5497fb534122f0278"
server_key_iv: "7037f577d4986da00d92559b5e6e4b05"
server_key_version: "encryptionkey@0"
I20260808 06:08:56.985531 2605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "198ef032fd9048e09d4d2436b0954f55"
format_stamp: "Formatted at 2026-08-08 06:08:56 on dist-test-slave-zj8q"
server_key: "3fbe0ac2a1b644a5497fb534122f0278"
server_key_iv: "7037f577d4986da00d92559b5e6e4b05"
server_key_version: "encryptionkey@0"
I20260808 06:08:56.989158 2605 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 06:08:56.991475 2629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:56.992619 2605 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 06:08:56.992743 2605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "198ef032fd9048e09d4d2436b0954f55"
format_stamp: "Formatted at 2026-08-08 06:08:56 on dist-test-slave-zj8q"
server_key: "3fbe0ac2a1b644a5497fb534122f0278"
server_key_iv: "7037f577d4986da00d92559b5e6e4b05"
server_key_version: "encryptionkey@0"
I20260808 06:08:56.992839 2605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:08:57.008801 2605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:08:57.011479 2605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:08:57.011648 2605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:08:57.018464 2605 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41799
I20260808 06:08:57.018472 2719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41799 every 8 connection(s)
I20260808 06:08:57.019367 2605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:08:57.021960 2720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:08:57.027257 2720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 198ef032fd9048e09d4d2436b0954f55: Bootstrap starting.
I20260808 06:08:57.028245 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 2605
I20260808 06:08:57.028369 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:08:57.028614 1630 external_mini_cluster.cc:1428] Setting key 159420e88b9c6e8f63559f1e38052852
I20260808 06:08:57.029778 2720 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 198ef032fd9048e09d4d2436b0954f55: Neither blocks nor log segments found. Creating new log.
I20260808 06:08:57.030498 2720 log.cc:824] T 00000000000000000000000000000000 P 198ef032fd9048e09d4d2436b0954f55: Log is configured to *not* fsync() on all Append() calls
I20260808 06:08:57.032377 2720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 198ef032fd9048e09d4d2436b0954f55: No bootstrap required, opened a new log
I20260808 06:08:57.036234 2720 raft_consensus.cc:348] T 00000000000000000000000000000000 P 198ef032fd9048e09d4d2436b0954f55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "198ef032fd9048e09d4d2436b0954f55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41799 } }
I20260808 06:08:57.036525 2720 raft_consensus.cc:374] T 00000000000000000000000000000000 P 198ef032fd9048e09d4d2436b0954f55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:08:57.036579 2720 raft_consensus.cc:729] T 00000000000000000000000000000000 P 198ef032fd9048e09d4d2436b0954f55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 198ef032fd9048e09d4d2436b0954f55, State: Initialized, Role: FOLLOWER
I20260808 06:08:57.037001 2720 consensus_queue.cc:260] T 00000000000000000000000000000000 P 198ef032fd9048e09d4d2436b0954f55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "198ef032fd9048e09d4d2436b0954f55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41799 } }
I20260808 06:08:57.037133 2720 raft_consensus.cc:388] T 00000000000000000000000000000000 P 198ef032fd9048e09d4d2436b0954f55 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:08:57.037204 2720 raft_consensus.cc:482] T 00000000000000000000000000000000 P 198ef032fd9048e09d4d2436b0954f55 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:08:57.037298 2720 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 198ef032fd9048e09d4d2436b0954f55 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:08:57.037324 2723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:57.029714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.236:43211 (local address 127.0.0.1:41799)
0808 06:08:57.030140 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:57.030206 (+ 66us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:57.030243 (+ 37us) server_negotiation.cc:407] Connection header received
0808 06:08:57.030889 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:57.030908 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:57.031254 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:57.031515 (+ 261us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:57.031901 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:57.032911 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:57.034144 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:57.034645 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:57.034763 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:57.035657 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:57.035663 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:57.035674 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:08:57.036031 (+ 357us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:57.036083 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:08:57.036105 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:57.036213 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:57.036395 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 06:08:57.036522 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":107,"threads_started":1}
I20260808 06:08:57.038309 2720 raft_consensus.cc:504] T 00000000000000000000000000000000 P 198ef032fd9048e09d4d2436b0954f55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "198ef032fd9048e09d4d2436b0954f55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41799 } }
I20260808 06:08:57.038753 2720 leader_election.cc:302] T 00000000000000000000000000000000 P 198ef032fd9048e09d4d2436b0954f55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 198ef032fd9048e09d4d2436b0954f55; no voters:
I20260808 06:08:57.039039 2720 leader_election.cc:288] T 00000000000000000000000000000000 P 198ef032fd9048e09d4d2436b0954f55 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:08:57.039108 2727 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 198ef032fd9048e09d4d2436b0954f55 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:08:57.039335 2727 raft_consensus.cc:686] T 00000000000000000000000000000000 P 198ef032fd9048e09d4d2436b0954f55 [term 1 LEADER]: Becoming Leader. State: Replica: 198ef032fd9048e09d4d2436b0954f55, State: Running, Role: LEADER
I20260808 06:08:57.039686 2727 consensus_queue.cc:237] T 00000000000000000000000000000000 P 198ef032fd9048e09d4d2436b0954f55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "198ef032fd9048e09d4d2436b0954f55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41799 } }
I20260808 06:08:57.040239 2720 sys_catalog.cc:564] T 00000000000000000000000000000000 P 198ef032fd9048e09d4d2436b0954f55 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:08:57.040340 1630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 06:08:57.041268 2729 sys_catalog.cc:455] T 00000000000000000000000000000000 P 198ef032fd9048e09d4d2436b0954f55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "198ef032fd9048e09d4d2436b0954f55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "198ef032fd9048e09d4d2436b0954f55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41799 } } }
I20260808 06:08:57.041389 2729 sys_catalog.cc:458] T 00000000000000000000000000000000 P 198ef032fd9048e09d4d2436b0954f55 [sys.catalog]: This master's current role is: LEADER
I20260808 06:08:57.041692 2732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 198ef032fd9048e09d4d2436b0954f55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 198ef032fd9048e09d4d2436b0954f55. Latest consensus state: current_term: 1 leader_uuid: "198ef032fd9048e09d4d2436b0954f55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "198ef032fd9048e09d4d2436b0954f55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41799 } } }
I20260808 06:08:57.041836 2732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 198ef032fd9048e09d4d2436b0954f55 [sys.catalog]: This master's current role is: LEADER
I20260808 06:08:57.042130 2745 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:08:57.045652 2745 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:08:57.047034 2723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:57.042604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.236:57805 (local address 127.0.0.1:41799)
0808 06:08:57.042695 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:57.042700 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:57.043092 (+ 392us) server_negotiation.cc:407] Connection header received
0808 06:08:57.043226 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:57.043228 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:57.043266 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:57.043328 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:57.043575 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:57.044132 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:57.045226 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:08:57.045559 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:08:57.045672 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:08:57.046539 (+ 867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:57.046543 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:57.046545 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:08:57.046559 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:57.046588 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:08:57.046601 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:57.046708 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:57.046867 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 06:08:57.046902 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 06:08:57.051432 2745 catalog_manager.cc:1347] Generated new cluster ID: 92dc4dbe811e48528b206e6174fc7331
I20260808 06:08:57.051573 2745 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:08:57.057236 2745 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:08:57.058470 2745 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:08:57.069355 2745 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 198ef032fd9048e09d4d2436b0954f55: Generated new TSK 0
I20260808 06:08:57.070070 2745 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 06:08:57.073133 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 2605
W20260808 06:08:57.087764 2578 connection.cc:535] client connection to 127.0.0.1:41799 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41799): Connection reset by peer (error 104)
2026-08-08T06:08:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1495 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:08:57 dist-test-slave-zj8q krb5kdc[2763](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:08:57 dist-test-slave-zj8q krb5kdc[2763](info): set up 2 sockets
Aug 08 06:08:57 dist-test-slave-zj8q krb5kdc[2763](info): commencing operation
krb5kdc: starting...
W20260808 06:08:58.254995 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:08:58 dist-test-slave-zj8q krb5kdc[2763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169338, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:08:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:08:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 06:08:58.371177 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44255
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44255
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:08:58.479869 2784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:58.480118 2784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:58.480157 2784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:58.483978 2784 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:08:58.484025 2784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:58.484047 2784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:58.484062 2784 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:08:58.484077 2784 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:08:58.489126 2784 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:58.490046 2784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:58.491000 2784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:08:58.498930 2797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:58.499076 2784 server_base.cc:1034] running on GCE node
W20260808 06:08:58.498947 2794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:58.498986 2793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:58.499722 2784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:58.500725 2784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:58.501886 2784 hybrid_clock.cc:648] HybridClock initialized: now 1786169338501866 us; error 32 us; skew 500 ppm
Aug 08 06:08:58 dist-test-slave-zj8q krb5kdc[2763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169338, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:08:58.504489 2784 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 06:08:58.505311 2784 webserver.cc:460] Webserver started at http://127.0.0.1:39673/ using document root <none> and password file <none>
I20260808 06:08:58.505848 2784 fs_manager.cc:359] Metadata directory not provided
I20260808 06:08:58.505915 2784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:08:58.506110 2784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:08:58.508086 2784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "09178bf07a014d4d971d9191e8ea1667"
format_stamp: "Formatted at 2026-08-08 06:08:58 on dist-test-slave-zj8q"
server_key: "55fc5e34b085b666e3f78860bd2d6ab8"
server_key_iv: "3e1e1cca979daec5d1fc8bcef1617273"
server_key_version: "encryptionkey@0"
I20260808 06:08:58.508618 2784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "09178bf07a014d4d971d9191e8ea1667"
format_stamp: "Formatted at 2026-08-08 06:08:58 on dist-test-slave-zj8q"
server_key: "55fc5e34b085b666e3f78860bd2d6ab8"
server_key_iv: "3e1e1cca979daec5d1fc8bcef1617273"
server_key_version: "encryptionkey@0"
I20260808 06:08:58.512210 2784 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 06:08:58.514518 2806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:08:58.515626 2784 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:08:58.515779 2784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "09178bf07a014d4d971d9191e8ea1667"
format_stamp: "Formatted at 2026-08-08 06:08:58 on dist-test-slave-zj8q"
server_key: "55fc5e34b085b666e3f78860bd2d6ab8"
server_key_iv: "3e1e1cca979daec5d1fc8bcef1617273"
server_key_version: "encryptionkey@0"
I20260808 06:08:58.515872 2784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:08:58.520252 2784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:08:58.523064 2784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:08:58.523222 2784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:08:58.529750 2784 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44255
I20260808 06:08:58.529795 2890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44255 every 8 connection(s)
I20260808 06:08:58.530629 2784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:08:58.533260 2891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:08:58.536610 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 2784
I20260808 06:08:58.536741 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:08:58.537006 1630 external_mini_cluster.cc:1428] Setting key 7fd6741e9aaf9c4cc9dda24a97074092
I20260808 06:08:58.539372 2891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09178bf07a014d4d971d9191e8ea1667: Bootstrap starting.
I20260808 06:08:58.542062 2891 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09178bf07a014d4d971d9191e8ea1667: Neither blocks nor log segments found. Creating new log.
I20260808 06:08:58.542364 2894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:58.538358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.236:54929 (local address 127.0.0.1:44255)
0808 06:08:58.538780 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:58.538797 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:58.538838 (+ 41us) server_negotiation.cc:407] Connection header received
0808 06:08:58.539651 (+ 813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:58.539675 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:58.540009 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:58.540363 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:58.541005 (+ 642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:58.541010 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:58.541031 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:08:58.541042 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:58.541102 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:08:58.541127 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:58.541255 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:58.541319 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 06:08:58.541465 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":111,"threads_started":1}
I20260808 06:08:58.542932 2891 log.cc:824] T 00000000000000000000000000000000 P 09178bf07a014d4d971d9191e8ea1667: Log is configured to *not* fsync() on all Append() calls
I20260808 06:08:58.544862 2891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09178bf07a014d4d971d9191e8ea1667: No bootstrap required, opened a new log
I20260808 06:08:58.545249 1630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 06:08:58.549073 2894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:08:58.547539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.236:46409 (local address 127.0.0.1:44255)
0808 06:08:58.547652 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 06:08:58.547656 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:08:58.547864 (+ 208us) server_negotiation.cc:407] Connection header received
0808 06:08:58.547938 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:08:58.547941 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:08:58.547978 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:08:58.548073 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:08:58.548723 (+ 650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:08:58.548726 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:08:58.548729 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:08:58.548741 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:08:58.548770 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:08:58.548784 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:08:58.548871 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 06:08:58.548916 (+ 45us) server_negotiation.cc:299] Negotiation successful
0808 06:08:58.548953 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 06:08:58.549219 2891 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09178bf07a014d4d971d9191e8ea1667 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09178bf07a014d4d971d9191e8ea1667" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44255 } }
I20260808 06:08:58.549446 2891 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09178bf07a014d4d971d9191e8ea1667 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:08:58.549502 2891 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09178bf07a014d4d971d9191e8ea1667 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09178bf07a014d4d971d9191e8ea1667, State: Initialized, Role: FOLLOWER
I20260808 06:08:58.549952 2891 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09178bf07a014d4d971d9191e8ea1667 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09178bf07a014d4d971d9191e8ea1667" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44255 } }
I20260808 06:08:58.550092 2891 raft_consensus.cc:388] T 00000000000000000000000000000000 P 09178bf07a014d4d971d9191e8ea1667 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:08:58.550174 2891 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09178bf07a014d4d971d9191e8ea1667 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:08:58.550272 2891 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09178bf07a014d4d971d9191e8ea1667 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:08:58.551128 2891 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09178bf07a014d4d971d9191e8ea1667 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09178bf07a014d4d971d9191e8ea1667" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44255 } }
I20260808 06:08:58.551446 2891 leader_election.cc:302] T 00000000000000000000000000000000 P 09178bf07a014d4d971d9191e8ea1667 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 09178bf07a014d4d971d9191e8ea1667; no voters:
I20260808 06:08:58.551729 2891 leader_election.cc:288] T 00000000000000000000000000000000 P 09178bf07a014d4d971d9191e8ea1667 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:08:58.551842 2906 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09178bf07a014d4d971d9191e8ea1667 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:08:58.552049 2906 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09178bf07a014d4d971d9191e8ea1667 [term 1 LEADER]: Becoming Leader. State: Replica: 09178bf07a014d4d971d9191e8ea1667, State: Running, Role: LEADER
I20260808 06:08:58.552376 2906 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09178bf07a014d4d971d9191e8ea1667 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09178bf07a014d4d971d9191e8ea1667" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44255 } }
I20260808 06:08:58.552873 2891 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09178bf07a014d4d971d9191e8ea1667 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:08:58.554023 2911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09178bf07a014d4d971d9191e8ea1667 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09178bf07a014d4d971d9191e8ea1667. Latest consensus state: current_term: 1 leader_uuid: "09178bf07a014d4d971d9191e8ea1667" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09178bf07a014d4d971d9191e8ea1667" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44255 } } }
I20260808 06:08:58.554028 2909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09178bf07a014d4d971d9191e8ea1667 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "09178bf07a014d4d971d9191e8ea1667" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09178bf07a014d4d971d9191e8ea1667" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44255 } } }
I20260808 06:08:58.554258 2911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09178bf07a014d4d971d9191e8ea1667 [sys.catalog]: This master's current role is: LEADER
I20260808 06:08:58.554280 2909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09178bf07a014d4d971d9191e8ea1667 [sys.catalog]: This master's current role is: LEADER
I20260808 06:08:58.554704 2923 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:08:58.557955 2923 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:08:58.563496 2923 catalog_manager.cc:1347] Generated new cluster ID: 9b694433d5e349359f253e15ab69574c
I20260808 06:08:58.563558 2923 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:08:58.582228 2923 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:08:58.582573 2923 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:08:58.590582 2923 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09178bf07a014d4d971d9191e8ea1667: Generated new TSK 0
I20260808 06:08:58.591429 2923 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 06:08:58.596043 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 2784
2026-08-08T06:08:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1526 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:08:58 dist-test-slave-zj8q krb5kdc[2940](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:08:58 dist-test-slave-zj8q krb5kdc[2940](info): set up 2 sockets
Aug 08 06:08:58 dist-test-slave-zj8q krb5kdc[2940](info): commencing operation
krb5kdc: starting...
W20260808 06:08:59.748868 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:08:59 dist-test-slave-zj8q krb5kdc[2940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:08:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:08:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 06:08:59.866274 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43723
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43723
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:08:59.973784 2961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:08:59.974021 2961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:08:59.974086 2961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:08:59.977931 2961 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:08:59.977988 2961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:08:59.978014 2961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:08:59.978041 2961 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:08:59.978070 2961 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:08:59.983103 2961 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43723
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:08:59.984058 2961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:08:59.985002 2961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:08:59.993036 2972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:59.993088 2969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:08:59.993088 2970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:08:59.993257 2961 server_base.cc:1034] running on GCE node
I20260808 06:08:59.993837 2961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:08:59.994870 2961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:08:59.996043 2961 hybrid_clock.cc:648] HybridClock initialized: now 1786169339996029 us; error 27 us; skew 500 ppm
Aug 08 06:08:59 dist-test-slave-zj8q krb5kdc[2940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169339, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:08:59.998646 2961 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 06:08:59.999439 2961 webserver.cc:460] Webserver started at http://127.0.0.1:45647/ using document root <none> and password file <none>
I20260808 06:08:59.999945 2961 fs_manager.cc:359] Metadata directory not provided
I20260808 06:09:00.000011 2961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:09:00.000268 2961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:09:00.002285 2961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "fb0187dca7dd4e6ca5cd86a54152e24c"
format_stamp: "Formatted at 2026-08-08 06:08:59 on dist-test-slave-zj8q"
server_key: "42710bc48d00848a2445a6e248ac4a06"
server_key_iv: "54918c0c8f30b93dd7a7177a78238ea7"
server_key_version: "encryptionkey@0"
I20260808 06:09:00.002825 2961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "fb0187dca7dd4e6ca5cd86a54152e24c"
format_stamp: "Formatted at 2026-08-08 06:08:59 on dist-test-slave-zj8q"
server_key: "42710bc48d00848a2445a6e248ac4a06"
server_key_iv: "54918c0c8f30b93dd7a7177a78238ea7"
server_key_version: "encryptionkey@0"
I20260808 06:09:00.006407 2961 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260808 06:09:00.008766 2981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:09:00.009933 2961 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:09:00.010048 2961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "fb0187dca7dd4e6ca5cd86a54152e24c"
format_stamp: "Formatted at 2026-08-08 06:08:59 on dist-test-slave-zj8q"
server_key: "42710bc48d00848a2445a6e248ac4a06"
server_key_iv: "54918c0c8f30b93dd7a7177a78238ea7"
server_key_version: "encryptionkey@0"
I20260808 06:09:00.010176 2961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:09:00.039552 2961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:09:00.042532 2961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:09:00.042728 2961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:09:00.049902 2961 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43723
I20260808 06:09:00.049932 3071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43723 every 8 connection(s)
I20260808 06:09:00.050848 2961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:09:00.051946 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 2961
I20260808 06:09:00.052073 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:09:00.052361 1630 external_mini_cluster.cc:1428] Setting key 685b21eea72aaea00e6f8cc86286602c
I20260808 06:09:00.053889 3072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:09:00.060233 3072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb0187dca7dd4e6ca5cd86a54152e24c: Bootstrap starting.
Aug 08 06:09:00 dist-test-slave-zj8q krb5kdc[2940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 06:09:00.063052 3072 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb0187dca7dd4e6ca5cd86a54152e24c: Neither blocks nor log segments found. Creating new log.
I20260808 06:09:00.063942 3072 log.cc:824] T 00000000000000000000000000000000 P fb0187dca7dd4e6ca5cd86a54152e24c: Log is configured to *not* fsync() on all Append() calls
I20260808 06:09:00.065840 3072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb0187dca7dd4e6ca5cd86a54152e24c: No bootstrap required, opened a new log
I20260808 06:09:00.067131 3076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:00.053731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.236:53799 (local address 127.0.0.1:43723)
0808 06:09:00.054188 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:00.054208 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:00.054249 (+ 41us) server_negotiation.cc:407] Connection header received
0808 06:09:00.054892 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:00.054909 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:00.055235 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:00.055501 (+ 266us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:00.056389 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:00.057405 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:00.058571 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:00.059031 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:00.059144 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:00.062295 (+ 3151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:00.062302 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:00.062318 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:09:00.062350 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:00.064307 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:00.064794 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:00.064800 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:00.064806 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:00.064919 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:00.065239 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:00.065243 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:00.065244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:00.065635 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:00.065824 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:00.066137 (+ 313us) server_negotiation.cc:299] Negotiation successful
0808 06:09:00.066441 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":142,"threads_started":1}
I20260808 06:09:00.069335 3072 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb0187dca7dd4e6ca5cd86a54152e24c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb0187dca7dd4e6ca5cd86a54152e24c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43723 } }
I20260808 06:09:00.069550 3072 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb0187dca7dd4e6ca5cd86a54152e24c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:09:00.069607 3072 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb0187dca7dd4e6ca5cd86a54152e24c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb0187dca7dd4e6ca5cd86a54152e24c, State: Initialized, Role: FOLLOWER
I20260808 06:09:00.069841 1630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 06:09:00.070048 3072 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb0187dca7dd4e6ca5cd86a54152e24c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb0187dca7dd4e6ca5cd86a54152e24c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43723 } }
I20260808 06:09:00.070223 3072 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb0187dca7dd4e6ca5cd86a54152e24c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:09:00.070286 3072 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb0187dca7dd4e6ca5cd86a54152e24c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:09:00.070381 3072 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb0187dca7dd4e6ca5cd86a54152e24c [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:09:00.071465 3072 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb0187dca7dd4e6ca5cd86a54152e24c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb0187dca7dd4e6ca5cd86a54152e24c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43723 } }
I20260808 06:09:00.071816 3072 leader_election.cc:302] T 00000000000000000000000000000000 P fb0187dca7dd4e6ca5cd86a54152e24c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb0187dca7dd4e6ca5cd86a54152e24c; no voters:
I20260808 06:09:00.072120 3072 leader_election.cc:288] T 00000000000000000000000000000000 P fb0187dca7dd4e6ca5cd86a54152e24c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:09:00.072290 3086 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb0187dca7dd4e6ca5cd86a54152e24c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:09:00.072477 3086 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb0187dca7dd4e6ca5cd86a54152e24c [term 1 LEADER]: Becoming Leader. State: Replica: fb0187dca7dd4e6ca5cd86a54152e24c, State: Running, Role: LEADER
I20260808 06:09:00.072765 3086 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb0187dca7dd4e6ca5cd86a54152e24c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb0187dca7dd4e6ca5cd86a54152e24c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43723 } }
I20260808 06:09:00.073375 3072 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb0187dca7dd4e6ca5cd86a54152e24c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:09:00.073915 3086 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb0187dca7dd4e6ca5cd86a54152e24c [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb0187dca7dd4e6ca5cd86a54152e24c. Latest consensus state: current_term: 1 leader_uuid: "fb0187dca7dd4e6ca5cd86a54152e24c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb0187dca7dd4e6ca5cd86a54152e24c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43723 } } }
I20260808 06:09:00.074036 3086 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb0187dca7dd4e6ca5cd86a54152e24c [sys.catalog]: This master's current role is: LEADER
I20260808 06:09:00.074527 3098 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:09:00.074723 3091 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb0187dca7dd4e6ca5cd86a54152e24c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb0187dca7dd4e6ca5cd86a54152e24c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb0187dca7dd4e6ca5cd86a54152e24c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43723 } } }
I20260808 06:09:00.074882 3091 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb0187dca7dd4e6ca5cd86a54152e24c [sys.catalog]: This master's current role is: LEADER
I20260808 06:09:00.077783 3098 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:09:00.081321 3076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:00.072053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.236:55939 (local address 127.0.0.1:43723)
0808 06:09:00.072167 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:00.072171 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:00.072376 (+ 205us) server_negotiation.cc:407] Connection header received
0808 06:09:00.072501 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:00.072505 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:00.072549 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:00.072597 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:00.073357 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:00.074104 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:00.075214 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:00.075529 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:00.075641 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:00.077733 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:00.077736 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:00.077739 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:09:00.077764 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:00.079166 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:00.079596 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:00.079600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:00.079602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:00.079679 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:00.080032 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:00.080036 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:00.080037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:00.080243 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:00.080373 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:00.081084 (+ 711us) server_negotiation.cc:299] Negotiation successful
0808 06:09:00.081180 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 06:09:00.083577 3098 catalog_manager.cc:1347] Generated new cluster ID: 4c86f916f98d455993f1b95fd858a02a
I20260808 06:09:00.083649 3098 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:09:00.114506 3098 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:09:00.115435 3098 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:09:00.119352 3098 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb0187dca7dd4e6ca5cd86a54152e24c: Generated new TSK 0
I20260808 06:09:00.120178 3098 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 06:09:00.123649 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 2961
W20260808 06:09:00.137401 2936 connection.cc:535] client connection to 127.0.0.1:43723 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43723): Connection reset by peer (error 104)
2026-08-08T06:09:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1523 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:09:00 dist-test-slave-zj8q krb5kdc[3124](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:09:00 dist-test-slave-zj8q krb5kdc[3124](info): set up 2 sockets
Aug 08 06:09:00 dist-test-slave-zj8q krb5kdc[3124](info): commencing operation
krb5kdc: starting...
W20260808 06:09:01.264485 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:09:01 dist-test-slave-zj8q krb5kdc[3124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:09:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:09:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 06:09:01.381793 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36713
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:09:01.489799 3153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:09:01.490029 3153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:09:01.490067 3153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:09:01.493845 3153 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:09:01.493892 3153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:09:01.493909 3153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:09:01.493924 3153 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:09:01.493937 3153 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:09:01.498893 3153 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36713
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:09:01.499804 3153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:09:01.500727 3153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:09:01.508744 3162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:09:01.508752 3159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:09:01.508806 3160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:09:01.509052 3153 server_base.cc:1034] running on GCE node
I20260808 06:09:01.509629 3153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:09:01.510692 3153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:09:01.511875 3153 hybrid_clock.cc:648] HybridClock initialized: now 1786169341511861 us; error 25 us; skew 500 ppm
Aug 08 06:09:01 dist-test-slave-zj8q krb5kdc[3124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169341, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:09:01.514585 3153 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 06:09:01.515389 3153 webserver.cc:460] Webserver started at http://127.0.0.1:36353/ using document root <none> and password file <none>
I20260808 06:09:01.515897 3153 fs_manager.cc:359] Metadata directory not provided
I20260808 06:09:01.515956 3153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:09:01.516166 3153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:09:01.518123 3153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "0e1879f105044191bb4f6b0ccdc185d5"
format_stamp: "Formatted at 2026-08-08 06:09:01 on dist-test-slave-zj8q"
server_key: "60187dbd515456cb9bc1bb7fea7c3667"
server_key_iv: "47dfe8325e96955a0de622e758d51231"
server_key_version: "encryptionkey@0"
I20260808 06:09:01.518687 3153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "0e1879f105044191bb4f6b0ccdc185d5"
format_stamp: "Formatted at 2026-08-08 06:09:01 on dist-test-slave-zj8q"
server_key: "60187dbd515456cb9bc1bb7fea7c3667"
server_key_iv: "47dfe8325e96955a0de622e758d51231"
server_key_version: "encryptionkey@0"
I20260808 06:09:01.522073 3153 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 06:09:01.524428 3177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:09:01.525651 3153 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:09:01.525760 3153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "0e1879f105044191bb4f6b0ccdc185d5"
format_stamp: "Formatted at 2026-08-08 06:09:01 on dist-test-slave-zj8q"
server_key: "60187dbd515456cb9bc1bb7fea7c3667"
server_key_iv: "47dfe8325e96955a0de622e758d51231"
server_key_version: "encryptionkey@0"
I20260808 06:09:01.525864 3153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:09:01.540416 3153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:09:01.543354 3153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:09:01.543515 3153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:09:01.550416 3153 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36713
I20260808 06:09:01.550457 3260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36713 every 8 connection(s)
I20260808 06:09:01.551332 3153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:09:01.553992 3261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:09:01.557360 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 3153
I20260808 06:09:01.557474 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:09:01.557760 1630 external_mini_cluster.cc:1428] Setting key 4a3257977b7e7ce1b1eb9155c0561c4d
I20260808 06:09:01.559648 3261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e1879f105044191bb4f6b0ccdc185d5: Bootstrap starting.
I20260808 06:09:01.561811 3261 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e1879f105044191bb4f6b0ccdc185d5: Neither blocks nor log segments found. Creating new log.
I20260808 06:09:01.562564 3261 log.cc:824] T 00000000000000000000000000000000 P 0e1879f105044191bb4f6b0ccdc185d5: Log is configured to *not* fsync() on all Append() calls
I20260808 06:09:01.564376 3261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e1879f105044191bb4f6b0ccdc185d5: No bootstrap required, opened a new log
I20260808 06:09:01.566738 3268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:01.559171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.236:57601 (local address 127.0.0.1:36713)
0808 06:09:01.559610 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:01.559626 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:01.559665 (+ 39us) server_negotiation.cc:407] Connection header received
0808 06:09:01.560318 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:01.560335 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:01.560677 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:01.560987 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:01.561409 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:01.562504 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:01.563437 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:01.563904 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:01.564073 (+ 169us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:01.564997 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:01.565002 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:01.565015 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:09:01.565400 (+ 385us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:01.565457 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:09:01.565481 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:01.565627 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:01.565835 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 06:09:01.565971 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":118,"threads_started":1}
I20260808 06:09:01.568094 3261 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e1879f105044191bb4f6b0ccdc185d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e1879f105044191bb4f6b0ccdc185d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36713 } }
I20260808 06:09:01.568359 3261 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e1879f105044191bb4f6b0ccdc185d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:09:01.568427 3261 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e1879f105044191bb4f6b0ccdc185d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e1879f105044191bb4f6b0ccdc185d5, State: Initialized, Role: FOLLOWER
I20260808 06:09:01.568895 3261 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e1879f105044191bb4f6b0ccdc185d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e1879f105044191bb4f6b0ccdc185d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36713 } }
I20260808 06:09:01.569038 3261 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e1879f105044191bb4f6b0ccdc185d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:09:01.569128 3261 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e1879f105044191bb4f6b0ccdc185d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:09:01.569235 3261 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e1879f105044191bb4f6b0ccdc185d5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:09:01.569813 1630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 06:09:01.570269 3261 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e1879f105044191bb4f6b0ccdc185d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e1879f105044191bb4f6b0ccdc185d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36713 } }
I20260808 06:09:01.570633 3261 leader_election.cc:302] T 00000000000000000000000000000000 P 0e1879f105044191bb4f6b0ccdc185d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e1879f105044191bb4f6b0ccdc185d5; no voters:
I20260808 06:09:01.570959 3261 leader_election.cc:288] T 00000000000000000000000000000000 P 0e1879f105044191bb4f6b0ccdc185d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:09:01.571069 3271 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e1879f105044191bb4f6b0ccdc185d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:09:01.571296 3271 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e1879f105044191bb4f6b0ccdc185d5 [term 1 LEADER]: Becoming Leader. State: Replica: 0e1879f105044191bb4f6b0ccdc185d5, State: Running, Role: LEADER
I20260808 06:09:01.571635 3271 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e1879f105044191bb4f6b0ccdc185d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e1879f105044191bb4f6b0ccdc185d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36713 } }
I20260808 06:09:01.572255 3261 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e1879f105044191bb4f6b0ccdc185d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:09:01.573318 3277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e1879f105044191bb4f6b0ccdc185d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e1879f105044191bb4f6b0ccdc185d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e1879f105044191bb4f6b0ccdc185d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36713 } } }
I20260808 06:09:01.573455 3277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e1879f105044191bb4f6b0ccdc185d5 [sys.catalog]: This master's current role is: LEADER
I20260808 06:09:01.573756 3279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e1879f105044191bb4f6b0ccdc185d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e1879f105044191bb4f6b0ccdc185d5. Latest consensus state: current_term: 1 leader_uuid: "0e1879f105044191bb4f6b0ccdc185d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e1879f105044191bb4f6b0ccdc185d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36713 } } }
I20260808 06:09:01.573908 3279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e1879f105044191bb4f6b0ccdc185d5 [sys.catalog]: This master's current role is: LEADER
I20260808 06:09:01.576895 3268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:01.572082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.236:33555 (local address 127.0.0.1:36713)
0808 06:09:01.572179 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:01.572183 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:01.573008 (+ 825us) server_negotiation.cc:407] Connection header received
0808 06:09:01.573213 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:01.573216 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:01.573265 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:01.573312 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:01.573729 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:01.574470 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:01.575333 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:01.575614 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:01.575704 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:01.576471 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:01.576474 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:01.576476 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:09:01.576487 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:01.576512 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:09:01.576522 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:01.576590 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:01.576736 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 06:09:01.576770 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260808 06:09:01.577490 3308 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0e1879f105044191bb4f6b0ccdc185d5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 06:09:01.577574 3308 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 06:09:01.577634 3296 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:09:01.578552 3296 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:09:01.583906 3296 catalog_manager.cc:1347] Generated new cluster ID: cd215f5b74e14f4f81d8776335615ece
I20260808 06:09:01.583963 3296 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:09:01.594764 3296 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:09:01.595645 3296 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:09:01.604557 3296 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e1879f105044191bb4f6b0ccdc185d5: Generated new TSK 0
I20260808 06:09:01.605154 3296 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 06:09:01.613090 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 3153
W20260808 06:09:01.625685 3120 connection.cc:535] client connection to 127.0.0.1:36713 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36713): Connection reset by peer (error 104)
2026-08-08T06:09:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1488 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:09:01 dist-test-slave-zj8q krb5kdc[3317](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:09:01 dist-test-slave-zj8q krb5kdc[3317](info): set up 2 sockets
Aug 08 06:09:01 dist-test-slave-zj8q krb5kdc[3317](info): commencing operation
krb5kdc: starting...
W20260808 06:09:02.754345 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:09:02 dist-test-slave-zj8q krb5kdc[3317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:09:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:09:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 06:09:02.873106 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42507
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42507
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:09:02.981606 3339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:09:02.981863 3339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:09:02.981904 3339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:09:02.986186 3339 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:09:02.986246 3339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:09:02.986265 3339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:09:02.986299 3339 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:09:02.986330 3339 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:09:02.991596 3339 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42507
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786169070948234-1630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:09:02.992558 3339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:09:02.993511 3339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:09:03.001677 3347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:09:03.001718 3339 server_base.cc:1034] running on GCE node
W20260808 06:09:03.001684 3348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:09:03.001732 3350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:09:03.002509 3339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:09:03.003515 3339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:09:03.004688 3339 hybrid_clock.cc:648] HybridClock initialized: now 1786169343004663 us; error 35 us; skew 500 ppm
Aug 08 06:09:03 dist-test-slave-zj8q krb5kdc[3317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169343, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:09:03.009910 3339 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 06:09:03.010818 3339 webserver.cc:460] Webserver started at http://127.0.0.1:44443/ using document root <none> and password file <none>
I20260808 06:09:03.011360 3339 fs_manager.cc:359] Metadata directory not provided
I20260808 06:09:03.011427 3339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:09:03.011632 3339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:09:03.013650 3339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "fa14a865400d4b97be2829a68e599c6a"
format_stamp: "Formatted at 2026-08-08 06:09:03 on dist-test-slave-zj8q"
server_key: "6913035d047d64912bbf5469d0656027"
server_key_iv: "6fd3ec29e422274970aea99beb81effb"
server_key_version: "encryptionkey@0"
I20260808 06:09:03.014232 3339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "fa14a865400d4b97be2829a68e599c6a"
format_stamp: "Formatted at 2026-08-08 06:09:03 on dist-test-slave-zj8q"
server_key: "6913035d047d64912bbf5469d0656027"
server_key_iv: "6fd3ec29e422274970aea99beb81effb"
server_key_version: "encryptionkey@0"
I20260808 06:09:03.017834 3339 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 06:09:03.020234 3366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:09:03.021401 3339 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:09:03.021543 3339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "fa14a865400d4b97be2829a68e599c6a"
format_stamp: "Formatted at 2026-08-08 06:09:03 on dist-test-slave-zj8q"
server_key: "6913035d047d64912bbf5469d0656027"
server_key_iv: "6fd3ec29e422274970aea99beb81effb"
server_key_version: "encryptionkey@0"
I20260808 06:09:03.021664 3339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:09:03.026059 3339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:09:03.028703 3339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:09:03.028861 3339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:09:03.035550 3339 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42507
I20260808 06:09:03.035565 3439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42507 every 8 connection(s)
I20260808 06:09:03.036412 3339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:09:03.038427 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 3339
I20260808 06:09:03.038573 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:09:03.038923 1630 external_mini_cluster.cc:1428] Setting key 433929772e574ebb01957e43fa4f4a0d
I20260808 06:09:03.039312 3442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:09:03.043920 3447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:03.040284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.236:43109 (local address 127.0.0.1:42507)
0808 06:09:03.040724 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:03.040739 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:03.040774 (+ 35us) server_negotiation.cc:407] Connection header received
0808 06:09:03.041467 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:03.041485 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:03.041751 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:03.042044 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:03.042824 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:03.042829 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:03.042840 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:09:03.042851 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:03.042903 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:09:03.042922 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:03.043048 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:03.043115 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 06:09:03.043280 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":135,"threads_started":1}
I20260808 06:09:03.045656 3442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa14a865400d4b97be2829a68e599c6a: Bootstrap starting.
I20260808 06:09:03.046657 1630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 06:09:03.048032 3442 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa14a865400d4b97be2829a68e599c6a: Neither blocks nor log segments found. Creating new log.
I20260808 06:09:03.048803 3442 log.cc:824] T 00000000000000000000000000000000 P fa14a865400d4b97be2829a68e599c6a: Log is configured to *not* fsync() on all Append() calls
I20260808 06:09:03.050055 3447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:03.048544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.236:41055 (local address 127.0.0.1:42507)
0808 06:09:03.048662 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:03.048666 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:03.048808 (+ 142us) server_negotiation.cc:407] Connection header received
0808 06:09:03.048900 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:03.048903 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:03.048938 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:03.049016 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:03.049653 (+ 637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:03.049656 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:03.049659 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:09:03.049671 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:03.049698 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:09:03.049710 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:03.049793 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:03.049899 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 06:09:03.049938 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 06:09:03.050967 3442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa14a865400d4b97be2829a68e599c6a: No bootstrap required, opened a new log
I20260808 06:09:03.054628 3442 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa14a865400d4b97be2829a68e599c6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa14a865400d4b97be2829a68e599c6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42507 } }
I20260808 06:09:03.054834 3442 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa14a865400d4b97be2829a68e599c6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:09:03.054893 3442 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa14a865400d4b97be2829a68e599c6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa14a865400d4b97be2829a68e599c6a, State: Initialized, Role: FOLLOWER
I20260808 06:09:03.055279 3442 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa14a865400d4b97be2829a68e599c6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa14a865400d4b97be2829a68e599c6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42507 } }
I20260808 06:09:03.055394 3442 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa14a865400d4b97be2829a68e599c6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:09:03.055454 3442 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa14a865400d4b97be2829a68e599c6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:09:03.055546 3442 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa14a865400d4b97be2829a68e599c6a [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:09:03.056470 3442 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa14a865400d4b97be2829a68e599c6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa14a865400d4b97be2829a68e599c6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42507 } }
I20260808 06:09:03.056787 3442 leader_election.cc:302] T 00000000000000000000000000000000 P fa14a865400d4b97be2829a68e599c6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa14a865400d4b97be2829a68e599c6a; no voters:
I20260808 06:09:03.057077 3442 leader_election.cc:288] T 00000000000000000000000000000000 P fa14a865400d4b97be2829a68e599c6a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:09:03.057174 3461 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa14a865400d4b97be2829a68e599c6a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:09:03.057379 3461 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa14a865400d4b97be2829a68e599c6a [term 1 LEADER]: Becoming Leader. State: Replica: fa14a865400d4b97be2829a68e599c6a, State: Running, Role: LEADER
I20260808 06:09:03.057715 3461 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa14a865400d4b97be2829a68e599c6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa14a865400d4b97be2829a68e599c6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42507 } }
I20260808 06:09:03.058261 3442 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa14a865400d4b97be2829a68e599c6a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:09:03.059737 3462 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa14a865400d4b97be2829a68e599c6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa14a865400d4b97be2829a68e599c6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa14a865400d4b97be2829a68e599c6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42507 } } }
I20260808 06:09:03.059757 3464 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa14a865400d4b97be2829a68e599c6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa14a865400d4b97be2829a68e599c6a. Latest consensus state: current_term: 1 leader_uuid: "fa14a865400d4b97be2829a68e599c6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa14a865400d4b97be2829a68e599c6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42507 } } }
I20260808 06:09:03.059854 3462 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa14a865400d4b97be2829a68e599c6a [sys.catalog]: This master's current role is: LEADER
I20260808 06:09:03.059856 3464 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa14a865400d4b97be2829a68e599c6a [sys.catalog]: This master's current role is: LEADER
I20260808 06:09:03.060609 3475 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:09:03.063670 3475 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:09:03.069037 3475 catalog_manager.cc:1347] Generated new cluster ID: ba96b973f70e4b8a920742a6cdf93890
I20260808 06:09:03.069196 3475 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:09:03.083850 3475 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:09:03.084136 3475 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:09:03.089529 3475 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa14a865400d4b97be2829a68e599c6a: Generated new TSK 0
I20260808 06:09:03.090315 3475 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 06:09:03.103441 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 3339
2026-08-08T06:09:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1493 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:09:03 dist-test-slave-zj8q krb5kdc[3497](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:09:03 dist-test-slave-zj8q krb5kdc[3497](info): set up 2 sockets
Aug 08 06:09:03 dist-test-slave-zj8q krb5kdc[3497](info): commencing operation
krb5kdc: starting...
W20260808 06:09:04.251276 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:09:04 dist-test-slave-zj8q krb5kdc[3497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169344, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:09:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:09:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.151.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
I20260808 06:09:04.370716 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:34547
--webserver_interface=127.1.151.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:45493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:34547
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:09:04.478802 3519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:09:04.479038 3519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:09:04.479092 3519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:09:04.482941 3519 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:09:04.482993 3519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:09:04.483019 3519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:09:04.483042 3519 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:09:04.483069 3519 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:09:04.488102 3519 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:45493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:34547
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:34547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:09:04.489051 3519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:09:04.489986 3519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:09:04.497969 3532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:09:04.498129 3519 server_base.cc:1034] running on GCE node
W20260808 06:09:04.497978 3536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:09:04.497960 3530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:09:04.498898 3519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:09:04.499907 3519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:09:04.501085 3519 hybrid_clock.cc:648] HybridClock initialized: now 1786169344501066 us; error 29 us; skew 500 ppm
Aug 08 06:09:04 dist-test-slave-zj8q krb5kdc[3497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169344, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:09:04.503922 3519 init.cc:377] Logged in from keytab as kudu/127.1.151.190@KRBTEST.COM (short username kudu)
I20260808 06:09:04.504793 3519 webserver.cc:460] Webserver started at http://127.1.151.190:44427/ using document root <none> and password file <none>
I20260808 06:09:04.505322 3519 fs_manager.cc:359] Metadata directory not provided
I20260808 06:09:04.505390 3519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:09:04.505662 3519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:09:04.507695 3519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "a0c7d29226a54babbea406266ed0f8f2"
format_stamp: "Formatted at 2026-08-08 06:09:04 on dist-test-slave-zj8q"
server_key: "43d9aa88a8f5f6220adbd5351420e560"
server_key_iv: "c13d2d574d557c3e11660633ea5f5ed8"
server_key_version: "encryptionkey@0"
I20260808 06:09:04.508255 3519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "a0c7d29226a54babbea406266ed0f8f2"
format_stamp: "Formatted at 2026-08-08 06:09:04 on dist-test-slave-zj8q"
server_key: "43d9aa88a8f5f6220adbd5351420e560"
server_key_iv: "c13d2d574d557c3e11660633ea5f5ed8"
server_key_version: "encryptionkey@0"
I20260808 06:09:04.511889 3519 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260808 06:09:04.514165 3544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:09:04.515326 3519 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 06:09:04.515451 3519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "a0c7d29226a54babbea406266ed0f8f2"
format_stamp: "Formatted at 2026-08-08 06:09:04 on dist-test-slave-zj8q"
server_key: "43d9aa88a8f5f6220adbd5351420e560"
server_key_iv: "c13d2d574d557c3e11660633ea5f5ed8"
server_key_version: "encryptionkey@0"
I20260808 06:09:04.515589 3519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:09:04.534595 3519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:09:04.537714 3519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:09:04.537912 3519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:09:04.549793 3519 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:34547
I20260808 06:09:04.549834 3628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:34547 every 8 connection(s)
I20260808 06:09:04.550756 3519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:09:04.553579 3631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:09:04.557221 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 3519
I20260808 06:09:04.557341 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:09:04.557654 1630 external_mini_cluster.cc:1428] Setting key 69f380a282dfdc0820f1ff1f3e0acf4a
I20260808 06:09:04.559348 3631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0c7d29226a54babbea406266ed0f8f2: Bootstrap starting.
I20260808 06:09:04.561761 3631 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0c7d29226a54babbea406266ed0f8f2: Neither blocks nor log segments found. Creating new log.
I20260808 06:09:04.562532 3631 log.cc:824] T 00000000000000000000000000000000 P a0c7d29226a54babbea406266ed0f8f2: Log is configured to *not* fsync() on all Append() calls
I20260808 06:09:04.564574 3631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0c7d29226a54babbea406266ed0f8f2: No bootstrap required, opened a new log
Aug 08 06:09:04 dist-test-slave-zj8q krb5kdc[3497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169344, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:09:04.568718 3631 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0c7d29226a54babbea406266ed0f8f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0c7d29226a54babbea406266ed0f8f2" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34547 } }
I20260808 06:09:04.568935 3631 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0c7d29226a54babbea406266ed0f8f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:09:04.568977 3631 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0c7d29226a54babbea406266ed0f8f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0c7d29226a54babbea406266ed0f8f2, State: Initialized, Role: FOLLOWER
I20260808 06:09:04.569440 3631 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0c7d29226a54babbea406266ed0f8f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0c7d29226a54babbea406266ed0f8f2" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34547 } }
I20260808 06:09:04.569607 3631 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0c7d29226a54babbea406266ed0f8f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:09:04.569689 3631 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0c7d29226a54babbea406266ed0f8f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:09:04.569787 3631 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0c7d29226a54babbea406266ed0f8f2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:09:04.570773 3631 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0c7d29226a54babbea406266ed0f8f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0c7d29226a54babbea406266ed0f8f2" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34547 } }
I20260808 06:09:04.571152 3631 leader_election.cc:302] T 00000000000000000000000000000000 P a0c7d29226a54babbea406266ed0f8f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0c7d29226a54babbea406266ed0f8f2; no voters:
I20260808 06:09:04.571480 3631 leader_election.cc:288] T 00000000000000000000000000000000 P a0c7d29226a54babbea406266ed0f8f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:09:04.571576 3638 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0c7d29226a54babbea406266ed0f8f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:09:04.571801 3638 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0c7d29226a54babbea406266ed0f8f2 [term 1 LEADER]: Becoming Leader. State: Replica: a0c7d29226a54babbea406266ed0f8f2, State: Running, Role: LEADER
I20260808 06:09:04.572147 3638 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0c7d29226a54babbea406266ed0f8f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0c7d29226a54babbea406266ed0f8f2" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34547 } }
I20260808 06:09:04.572324 3634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:04.559056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41631 (local address 127.1.151.190:34547)
0808 06:09:04.559440 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:04.559459 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:04.559502 (+ 43us) server_negotiation.cc:407] Connection header received
0808 06:09:04.560134 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:04.560151 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:04.560471 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:04.560735 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:04.561666 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:04.562798 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:04.563966 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:04.564438 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:04.564563 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:04.567380 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:04.567386 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:04.567399 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:09:04.567427 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:04.569089 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:04.569639 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:04.569645 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:04.569651 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:04.569793 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:04.570133 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:04.570136 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:04.570137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:04.570535 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:04.570701 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:04.571029 (+ 328us) server_negotiation.cc:299] Negotiation successful
0808 06:09:04.571292 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":102,"threads_started":1}
I20260808 06:09:04.572691 3631 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0c7d29226a54babbea406266ed0f8f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:09:04.574867 3642 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0c7d29226a54babbea406266ed0f8f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0c7d29226a54babbea406266ed0f8f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0c7d29226a54babbea406266ed0f8f2" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34547 } } }
I20260808 06:09:04.575738 3642 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0c7d29226a54babbea406266ed0f8f2 [sys.catalog]: This master's current role is: LEADER
I20260808 06:09:04.575074 3640 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0c7d29226a54babbea406266ed0f8f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0c7d29226a54babbea406266ed0f8f2. Latest consensus state: current_term: 1 leader_uuid: "a0c7d29226a54babbea406266ed0f8f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0c7d29226a54babbea406266ed0f8f2" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 34547 } } }
I20260808 06:09:04.575958 3640 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0c7d29226a54babbea406266ed0f8f2 [sys.catalog]: This master's current role is: LEADER
I20260808 06:09:04.576211 1630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260808 06:09:04.577389 3658 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a0c7d29226a54babbea406266ed0f8f2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 06:09:04.577468 3658 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 06:09:04.577584 3661 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:09:04.578480 3661 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:09:04.583709 3661 catalog_manager.cc:1347] Generated new cluster ID: 492d63c4e56a40caafe2527567b609ee
I20260808 06:09:04.583765 3661 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:09:04.590831 3634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:04.578108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34035 (local address 127.1.151.190:34547)
0808 06:09:04.578236 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:04.578240 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:04.582465 (+ 4225us) server_negotiation.cc:407] Connection header received
0808 06:09:04.582585 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:04.582588 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:04.582640 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:04.582722 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:04.583531 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:04.584329 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:04.585528 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:04.585861 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:04.585953 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:04.587568 (+ 1615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:04.587572 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:04.587574 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:09:04.587598 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:04.589107 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:04.589633 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:04.589636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:04.589638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:04.589711 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:04.589996 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:04.589999 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:04.590000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:04.590220 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:04.590342 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:04.590590 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 06:09:04.590680 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 06:09:04.593441 3661 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:09:04.594347 3661 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:09:04.598814 3661 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0c7d29226a54babbea406266ed0f8f2: Generated new TSK 0
I20260808 06:09:04.599391 3661 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 06:09:04.604372 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 3519
W20260808 06:09:04.614478 3494 connection.cc:535] client connection to 127.1.151.190:34547 recv error: Network error: failed to read from TLS socket (remote: 127.1.151.190:34547): Connection reset by peer (error 104)
2026-08-08T06:09:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1494 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:09:04 dist-test-slave-zj8q krb5kdc[3673](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:09:04 dist-test-slave-zj8q krb5kdc[3673](info): set up 2 sockets
Aug 08 06:09:04 dist-test-slave-zj8q krb5kdc[3673](info): commencing operation
krb5kdc: starting...
W20260808 06:09:05.750097 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:09:05 dist-test-slave-zj8q krb5kdc[3673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:09:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:09:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.151.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
I20260808 06:09:05.870225 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:33957
--webserver_interface=127.1.151.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:40453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:33957
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:09:05.978809 3688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:09:05.979068 3688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:09:05.979110 3688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:09:05.982980 3688 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:09:05.983035 3688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:09:05.983053 3688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:09:05.983068 3688 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:09:05.983081 3688 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:09:05.988132 3688 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:40453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:33957
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:33957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:09:05.989008 3688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:09:05.989928 3688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:09:05.998095 3698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:09:05.998248 3688 server_base.cc:1034] running on GCE node
W20260808 06:09:05.998107 3694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:09:05.998107 3695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:09:05.998934 3688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:09:05.999982 3688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:09:06.001168 3688 hybrid_clock.cc:648] HybridClock initialized: now 1786169346001153 us; error 24 us; skew 500 ppm
Aug 08 06:09:06 dist-test-slave-zj8q krb5kdc[3673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169346, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:09:06.003810 3688 init.cc:377] Logged in from keytab as kudu/127.1.151.190@KRBTEST.COM (short username kudu)
I20260808 06:09:06.004640 3688 webserver.cc:460] Webserver started at http://127.1.151.190:39647/ using document root <none> and password file <none>
I20260808 06:09:06.005168 3688 fs_manager.cc:359] Metadata directory not provided
I20260808 06:09:06.005235 3688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:09:06.005440 3688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:09:06.007485 3688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "c5dfcd92ceee42b1bc1d9c941c5be856"
format_stamp: "Formatted at 2026-08-08 06:09:06 on dist-test-slave-zj8q"
server_key: "387477c65804102bf8da7bc5b2b91253"
server_key_iv: "e9180ee50abc18a19403e8c78e7e110f"
server_key_version: "encryptionkey@0"
I20260808 06:09:06.008030 3688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "c5dfcd92ceee42b1bc1d9c941c5be856"
format_stamp: "Formatted at 2026-08-08 06:09:06 on dist-test-slave-zj8q"
server_key: "387477c65804102bf8da7bc5b2b91253"
server_key_iv: "e9180ee50abc18a19403e8c78e7e110f"
server_key_version: "encryptionkey@0"
I20260808 06:09:06.011703 3688 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 06:09:06.015615 3709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:09:06.017283 3688 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260808 06:09:06.017455 3688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "c5dfcd92ceee42b1bc1d9c941c5be856"
format_stamp: "Formatted at 2026-08-08 06:09:06 on dist-test-slave-zj8q"
server_key: "387477c65804102bf8da7bc5b2b91253"
server_key_iv: "e9180ee50abc18a19403e8c78e7e110f"
server_key_version: "encryptionkey@0"
I20260808 06:09:06.017580 3688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:09:06.031098 3688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:09:06.034343 3688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:09:06.034523 3688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:09:06.041335 3688 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:33957
I20260808 06:09:06.041373 3787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:33957 every 8 connection(s)
I20260808 06:09:06.042279 3688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:09:06.044943 3788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:09:06.045845 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 3688
I20260808 06:09:06.045982 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:09:06.046325 1630 external_mini_cluster.cc:1428] Setting key 125e5dec722e3a01d2f051ef98933879
I20260808 06:09:06.050849 3788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5dfcd92ceee42b1bc1d9c941c5be856: Bootstrap starting.
I20260808 06:09:06.053040 3788 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5dfcd92ceee42b1bc1d9c941c5be856: Neither blocks nor log segments found. Creating new log.
I20260808 06:09:06.053758 3788 log.cc:824] T 00000000000000000000000000000000 P c5dfcd92ceee42b1bc1d9c941c5be856: Log is configured to *not* fsync() on all Append() calls
I20260808 06:09:06.055132 3793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:06.047658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34119 (local address 127.1.151.190:33957)
0808 06:09:06.048161 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:06.048176 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:06.048213 (+ 37us) server_negotiation.cc:407] Connection header received
0808 06:09:06.048841 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:06.048860 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:06.049198 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:06.049508 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:06.049911 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:06.050958 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:06.051872 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:06.052349 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:06.052493 (+ 144us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:06.053371 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:06.053376 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:06.053389 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:09:06.053761 (+ 372us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:06.053816 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:09:06.053839 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:06.053963 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:06.054179 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 06:09:06.054320 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":151,"threads_started":1}
I20260808 06:09:06.055799 3788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5dfcd92ceee42b1bc1d9c941c5be856: No bootstrap required, opened a new log
I20260808 06:09:06.058008 1630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 06:09:06.059837 3788 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5dfcd92ceee42b1bc1d9c941c5be856 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5dfcd92ceee42b1bc1d9c941c5be856" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 33957 } }
I20260808 06:09:06.060113 3788 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5dfcd92ceee42b1bc1d9c941c5be856 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:09:06.060173 3788 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5dfcd92ceee42b1bc1d9c941c5be856 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5dfcd92ceee42b1bc1d9c941c5be856, State: Initialized, Role: FOLLOWER
I20260808 06:09:06.060751 3788 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5dfcd92ceee42b1bc1d9c941c5be856 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5dfcd92ceee42b1bc1d9c941c5be856" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 33957 } }
I20260808 06:09:06.060917 3788 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5dfcd92ceee42b1bc1d9c941c5be856 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:09:06.060992 3788 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5dfcd92ceee42b1bc1d9c941c5be856 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:09:06.061092 3788 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5dfcd92ceee42b1bc1d9c941c5be856 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:09:06.062039 3788 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5dfcd92ceee42b1bc1d9c941c5be856 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5dfcd92ceee42b1bc1d9c941c5be856" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 33957 } }
I20260808 06:09:06.062428 3788 leader_election.cc:302] T 00000000000000000000000000000000 P c5dfcd92ceee42b1bc1d9c941c5be856 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5dfcd92ceee42b1bc1d9c941c5be856; no voters:
I20260808 06:09:06.062779 3788 leader_election.cc:288] T 00000000000000000000000000000000 P c5dfcd92ceee42b1bc1d9c941c5be856 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:09:06.062915 3806 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5dfcd92ceee42b1bc1d9c941c5be856 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:09:06.063120 3806 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5dfcd92ceee42b1bc1d9c941c5be856 [term 1 LEADER]: Becoming Leader. State: Replica: c5dfcd92ceee42b1bc1d9c941c5be856, State: Running, Role: LEADER
I20260808 06:09:06.063443 3806 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5dfcd92ceee42b1bc1d9c941c5be856 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5dfcd92ceee42b1bc1d9c941c5be856" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 33957 } }
I20260808 06:09:06.064033 3788 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5dfcd92ceee42b1bc1d9c941c5be856 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:09:06.064597 3793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:06.060248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36643 (local address 127.1.151.190:33957)
0808 06:09:06.060352 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:06.060356 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:06.060647 (+ 291us) server_negotiation.cc:407] Connection header received
0808 06:09:06.060736 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:06.060739 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:06.060785 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:06.060870 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:06.061146 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:06.061719 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:06.062844 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:06.063160 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:06.063717 (+ 557us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:06.064141 (+ 424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:06.064144 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:06.064147 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:09:06.064160 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:06.064188 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:09:06.064200 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:06.064286 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:06.064429 (+ 143us) server_negotiation.cc:299] Negotiation successful
0808 06:09:06.064463 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 06:09:06.065550 3810 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5dfcd92ceee42b1bc1d9c941c5be856 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5dfcd92ceee42b1bc1d9c941c5be856. Latest consensus state: current_term: 1 leader_uuid: "c5dfcd92ceee42b1bc1d9c941c5be856" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5dfcd92ceee42b1bc1d9c941c5be856" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 33957 } } }
I20260808 06:09:06.065587 3809 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5dfcd92ceee42b1bc1d9c941c5be856 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5dfcd92ceee42b1bc1d9c941c5be856" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5dfcd92ceee42b1bc1d9c941c5be856" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 33957 } } }
I20260808 06:09:06.065732 3810 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5dfcd92ceee42b1bc1d9c941c5be856 [sys.catalog]: This master's current role is: LEADER
I20260808 06:09:06.065764 3809 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5dfcd92ceee42b1bc1d9c941c5be856 [sys.catalog]: This master's current role is: LEADER
I20260808 06:09:06.066749 3821 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:09:06.069217 3821 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:09:06.074487 3821 catalog_manager.cc:1347] Generated new cluster ID: 1e38a887f0ac4d04b64268d019d6f5e0
I20260808 06:09:06.074548 3821 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:09:06.083389 3821 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:09:06.084563 3821 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:09:06.090487 3821 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5dfcd92ceee42b1bc1d9c941c5be856: Generated new TSK 0
I20260808 06:09:06.091094 3821 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 06:09:06.100687 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 3688
W20260808 06:09:06.114136 3671 connection.cc:535] client connection to 127.1.151.190:33957 recv error: Network error: failed to read from TLS socket (remote: 127.1.151.190:33957): Connection reset by peer (error 104)
2026-08-08T06:09:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1499 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:09:06 dist-test-slave-zj8q krb5kdc[3843](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:09:06 dist-test-slave-zj8q krb5kdc[3843](info): set up 2 sockets
Aug 08 06:09:06 dist-test-slave-zj8q krb5kdc[3843](info): commencing operation
krb5kdc: starting...
W20260808 06:09:07.269795 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:09:07 dist-test-slave-zj8q krb5kdc[3843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:09:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:09:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.151.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
I20260808 06:09:07.401249 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:38015
--webserver_interface=127.1.151.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:40795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:38015
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:09:07.512262 3866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:09:07.512498 3866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:09:07.512543 3866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:09:07.516489 3866 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:09:07.516541 3866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:09:07.516559 3866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:09:07.516590 3866 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:09:07.516618 3866 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:09:07.521704 3866 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:40795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:38015
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:38015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:09:07.522708 3866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:09:07.523658 3866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:09:07.531903 3873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:09:07.531976 3866 server_base.cc:1034] running on GCE node
W20260808 06:09:07.531919 3874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:09:07.532115 3877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:09:07.532720 3866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:09:07.533743 3866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:09:07.534912 3866 hybrid_clock.cc:648] HybridClock initialized: now 1786169347534887 us; error 38 us; skew 500 ppm
Aug 08 06:09:07 dist-test-slave-zj8q krb5kdc[3843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169347, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:09:07.537560 3866 init.cc:377] Logged in from keytab as kudu/127.1.151.190@KRBTEST.COM (short username kudu)
I20260808 06:09:07.538427 3866 webserver.cc:460] Webserver started at http://127.1.151.190:38203/ using document root <none> and password file <none>
I20260808 06:09:07.538941 3866 fs_manager.cc:359] Metadata directory not provided
I20260808 06:09:07.539006 3866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:09:07.539208 3866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:09:07.541158 3866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "ee538a7f3df54959990d5c246603e20a"
format_stamp: "Formatted at 2026-08-08 06:09:07 on dist-test-slave-zj8q"
server_key: "8705b1eb074ce8d12a62210b371a8ce7"
server_key_iv: "5ffa4572cbb41e2d018700d75f09e0f7"
server_key_version: "encryptionkey@0"
I20260808 06:09:07.541679 3866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "ee538a7f3df54959990d5c246603e20a"
format_stamp: "Formatted at 2026-08-08 06:09:07 on dist-test-slave-zj8q"
server_key: "8705b1eb074ce8d12a62210b371a8ce7"
server_key_iv: "5ffa4572cbb41e2d018700d75f09e0f7"
server_key_version: "encryptionkey@0"
I20260808 06:09:07.545179 3866 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 06:09:07.547509 3891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:09:07.548804 3866 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 06:09:07.548952 3866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "ee538a7f3df54959990d5c246603e20a"
format_stamp: "Formatted at 2026-08-08 06:09:07 on dist-test-slave-zj8q"
server_key: "8705b1eb074ce8d12a62210b371a8ce7"
server_key_iv: "5ffa4572cbb41e2d018700d75f09e0f7"
server_key_version: "encryptionkey@0"
I20260808 06:09:07.549048 3866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:09:07.553908 3866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:09:07.556944 3866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:09:07.557129 3866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:09:07.563812 3866 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:38015
I20260808 06:09:07.563859 3967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:38015 every 8 connection(s)
I20260808 06:09:07.564716 3866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:09:07.567232 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 3866
I20260808 06:09:07.567379 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:09:07.567555 3973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:09:07.567700 1630 external_mini_cluster.cc:1428] Setting key ad2f9bc12d66c2fb00480b211d30a6cd
I20260808 06:09:07.573110 3982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:07.569061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33509 (local address 127.1.151.190:38015)
0808 06:09:07.569514 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:07.569533 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:07.569574 (+ 41us) server_negotiation.cc:407] Connection header received
0808 06:09:07.570380 (+ 806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:07.570406 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:07.570759 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:07.571111 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:07.571753 (+ 642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:07.571759 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:07.571780 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:09:07.571793 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:07.571853 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:09:07.571879 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:07.572007 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:07.572069 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 06:09:07.572221 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":135,"threads_started":1}
I20260808 06:09:07.574729 3973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee538a7f3df54959990d5c246603e20a: Bootstrap starting.
I20260808 06:09:07.575946 1630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 06:09:07.576954 3973 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee538a7f3df54959990d5c246603e20a: Neither blocks nor log segments found. Creating new log.
I20260808 06:09:07.577725 3973 log.cc:824] T 00000000000000000000000000000000 P ee538a7f3df54959990d5c246603e20a: Log is configured to *not* fsync() on all Append() calls
I20260808 06:09:07.579512 3982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:07.577902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58187 (local address 127.1.151.190:38015)
0808 06:09:07.578008 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:07.578012 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:07.578199 (+ 187us) server_negotiation.cc:407] Connection header received
0808 06:09:07.578338 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:07.578342 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:07.578373 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:07.578434 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:07.579161 (+ 727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:07.579165 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:07.579167 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:09:07.579179 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:07.579205 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:09:07.579215 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:07.579283 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:07.579359 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 06:09:07.579395 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 06:09:07.579555 3973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee538a7f3df54959990d5c246603e20a: No bootstrap required, opened a new log
I20260808 06:09:07.583112 3973 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee538a7f3df54959990d5c246603e20a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee538a7f3df54959990d5c246603e20a" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 38015 } }
I20260808 06:09:07.583366 3973 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee538a7f3df54959990d5c246603e20a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:09:07.583422 3973 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee538a7f3df54959990d5c246603e20a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee538a7f3df54959990d5c246603e20a, State: Initialized, Role: FOLLOWER
I20260808 06:09:07.583813 3973 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee538a7f3df54959990d5c246603e20a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee538a7f3df54959990d5c246603e20a" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 38015 } }
I20260808 06:09:07.583942 3973 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee538a7f3df54959990d5c246603e20a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:09:07.584004 3973 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee538a7f3df54959990d5c246603e20a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:09:07.584100 3973 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee538a7f3df54959990d5c246603e20a [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:09:07.584933 3973 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee538a7f3df54959990d5c246603e20a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee538a7f3df54959990d5c246603e20a" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 38015 } }
I20260808 06:09:07.585249 3973 leader_election.cc:302] T 00000000000000000000000000000000 P ee538a7f3df54959990d5c246603e20a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee538a7f3df54959990d5c246603e20a; no voters:
I20260808 06:09:07.585532 3973 leader_election.cc:288] T 00000000000000000000000000000000 P ee538a7f3df54959990d5c246603e20a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:09:07.585652 3996 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee538a7f3df54959990d5c246603e20a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:09:07.585846 3996 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee538a7f3df54959990d5c246603e20a [term 1 LEADER]: Becoming Leader. State: Replica: ee538a7f3df54959990d5c246603e20a, State: Running, Role: LEADER
I20260808 06:09:07.586145 3996 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee538a7f3df54959990d5c246603e20a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee538a7f3df54959990d5c246603e20a" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 38015 } }
I20260808 06:09:07.586714 3973 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee538a7f3df54959990d5c246603e20a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:09:07.587805 3998 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee538a7f3df54959990d5c246603e20a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee538a7f3df54959990d5c246603e20a. Latest consensus state: current_term: 1 leader_uuid: "ee538a7f3df54959990d5c246603e20a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee538a7f3df54959990d5c246603e20a" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 38015 } } }
I20260808 06:09:07.587929 3998 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee538a7f3df54959990d5c246603e20a [sys.catalog]: This master's current role is: LEADER
I20260808 06:09:07.588330 3997 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee538a7f3df54959990d5c246603e20a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee538a7f3df54959990d5c246603e20a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee538a7f3df54959990d5c246603e20a" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 38015 } } }
I20260808 06:09:07.588479 3997 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee538a7f3df54959990d5c246603e20a [sys.catalog]: This master's current role is: LEADER
W20260808 06:09:07.591342 4017 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ee538a7f3df54959990d5c246603e20a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 06:09:07.591409 4017 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 06:09:07.591495 4009 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:09:07.592398 4009 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:09:07.597656 4009 catalog_manager.cc:1347] Generated new cluster ID: 4993290cf79749f8914a81f1e13f30ee
I20260808 06:09:07.597728 4009 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:09:07.617440 4009 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:09:07.617755 4009 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:09:07.627592 4009 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee538a7f3df54959990d5c246603e20a: Generated new TSK 0
I20260808 06:09:07.628304 4009 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 06:09:07.634284 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 3866
2026-08-08T06:09:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1536 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:09:07 dist-test-slave-zj8q krb5kdc[4028](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:09:07 dist-test-slave-zj8q krb5kdc[4028](info): set up 2 sockets
Aug 08 06:09:07 dist-test-slave-zj8q krb5kdc[4028](info): commencing operation
krb5kdc: starting...
W20260808 06:09:08.782099 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:09:08 dist-test-slave-zj8q krb5kdc[4028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:09:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:09:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.151.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
I20260808 06:09:08.898797 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:43757
--webserver_interface=127.1.151.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:44177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:43757
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:09:09.007797 4048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:09:09.008030 4048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:09:09.008069 4048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:09:09.011941 4048 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:09:09.011992 4048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:09:09.012010 4048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:09:09.012025 4048 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:09:09.012038 4048 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:09:09.017086 4048 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:44177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:43757
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:43757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:09:09.017989 4048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:09:09.018980 4048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:09:09.027256 4055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:09:09.027274 4053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:09:09.027266 4058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:09:09.027351 4048 server_base.cc:1034] running on GCE node
I20260808 06:09:09.028033 4048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:09:09.029138 4048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:09:09.030303 4048 hybrid_clock.cc:648] HybridClock initialized: now 1786169349030286 us; error 26 us; skew 500 ppm
Aug 08 06:09:09 dist-test-slave-zj8q krb5kdc[4028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169349, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:09:09.035766 4048 init.cc:377] Logged in from keytab as kudu/127.1.151.190@KRBTEST.COM (short username kudu)
I20260808 06:09:09.036693 4048 webserver.cc:460] Webserver started at http://127.1.151.190:40431/ using document root <none> and password file <none>
I20260808 06:09:09.037236 4048 fs_manager.cc:359] Metadata directory not provided
I20260808 06:09:09.037297 4048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:09:09.037470 4048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:09:09.039553 4048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "52e3d23012ac4bec811768c79781b476"
format_stamp: "Formatted at 2026-08-08 06:09:09 on dist-test-slave-zj8q"
server_key: "756a6cd3bee421253b72655b62b0960d"
server_key_iv: "0e84c3aabf9fc3b0e45d8604aaf606fa"
server_key_version: "encryptionkey@0"
I20260808 06:09:09.040092 4048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "52e3d23012ac4bec811768c79781b476"
format_stamp: "Formatted at 2026-08-08 06:09:09 on dist-test-slave-zj8q"
server_key: "756a6cd3bee421253b72655b62b0960d"
server_key_iv: "0e84c3aabf9fc3b0e45d8604aaf606fa"
server_key_version: "encryptionkey@0"
I20260808 06:09:09.043836 4048 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 06:09:09.046306 4067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:09:09.047533 4048 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:09:09.047675 4048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "52e3d23012ac4bec811768c79781b476"
format_stamp: "Formatted at 2026-08-08 06:09:09 on dist-test-slave-zj8q"
server_key: "756a6cd3bee421253b72655b62b0960d"
server_key_iv: "0e84c3aabf9fc3b0e45d8604aaf606fa"
server_key_version: "encryptionkey@0"
I20260808 06:09:09.047781 4048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:09:09.071419 4048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:09:09.074227 4048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:09:09.074406 4048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:09:09.081331 4048 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:43757
I20260808 06:09:09.081350 4150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:43757 every 8 connection(s)
I20260808 06:09:09.082304 4048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:09:09.084342 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 4048
I20260808 06:09:09.084463 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:09:09.084740 1630 external_mini_cluster.cc:1428] Setting key 5f4046f994ce0b0f11584f71489abc27
I20260808 06:09:09.085201 4152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:09:09.092650 4152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52e3d23012ac4bec811768c79781b476: Bootstrap starting.
Aug 08 06:09:09 dist-test-slave-zj8q krb5kdc[4028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:09:09.095155 4152 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52e3d23012ac4bec811768c79781b476: Neither blocks nor log segments found. Creating new log.
I20260808 06:09:09.095952 4152 log.cc:824] T 00000000000000000000000000000000 P 52e3d23012ac4bec811768c79781b476: Log is configured to *not* fsync() on all Append() calls
I20260808 06:09:09.098055 4152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52e3d23012ac4bec811768c79781b476: No bootstrap required, opened a new log
I20260808 06:09:09.099370 4159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:09.086010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50165 (local address 127.1.151.190:43757)
0808 06:09:09.086439 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:09.086454 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:09.086490 (+ 36us) server_negotiation.cc:407] Connection header received
0808 06:09:09.087131 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:09.087149 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:09.087462 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:09.087730 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:09.088732 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:09.090232 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:09.091252 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:09.092000 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:09.092177 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:09.094682 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:09.094688 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:09.094701 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:09:09.094728 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:09.096489 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:09.096962 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:09.096969 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:09.096974 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:09.097112 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:09.097378 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:09.097381 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:09.097383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:09.097872 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:09.098081 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:09.098373 (+ 292us) server_negotiation.cc:299] Negotiation successful
0808 06:09:09.098673 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":153,"threads_started":1}
I20260808 06:09:09.101689 4152 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52e3d23012ac4bec811768c79781b476 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52e3d23012ac4bec811768c79781b476" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 43757 } }
I20260808 06:09:09.101905 4152 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52e3d23012ac4bec811768c79781b476 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:09:09.101965 4152 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52e3d23012ac4bec811768c79781b476 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52e3d23012ac4bec811768c79781b476, State: Initialized, Role: FOLLOWER
I20260808 06:09:09.102125 1630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 06:09:09.102398 4152 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52e3d23012ac4bec811768c79781b476 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52e3d23012ac4bec811768c79781b476" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 43757 } }
I20260808 06:09:09.102516 4152 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52e3d23012ac4bec811768c79781b476 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:09:09.102555 4152 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52e3d23012ac4bec811768c79781b476 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:09:09.102617 4152 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52e3d23012ac4bec811768c79781b476 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:09:09.103472 4152 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52e3d23012ac4bec811768c79781b476 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52e3d23012ac4bec811768c79781b476" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 43757 } }
I20260808 06:09:09.103770 4152 leader_election.cc:302] T 00000000000000000000000000000000 P 52e3d23012ac4bec811768c79781b476 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52e3d23012ac4bec811768c79781b476; no voters:
I20260808 06:09:09.104089 4152 leader_election.cc:288] T 00000000000000000000000000000000 P 52e3d23012ac4bec811768c79781b476 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:09:09.104272 4167 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52e3d23012ac4bec811768c79781b476 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:09:09.104542 4167 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52e3d23012ac4bec811768c79781b476 [term 1 LEADER]: Becoming Leader. State: Replica: 52e3d23012ac4bec811768c79781b476, State: Running, Role: LEADER
I20260808 06:09:09.104905 4167 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52e3d23012ac4bec811768c79781b476 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52e3d23012ac4bec811768c79781b476" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 43757 } }
I20260808 06:09:09.105288 4152 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52e3d23012ac4bec811768c79781b476 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:09:09.106211 4167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52e3d23012ac4bec811768c79781b476 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52e3d23012ac4bec811768c79781b476. Latest consensus state: current_term: 1 leader_uuid: "52e3d23012ac4bec811768c79781b476" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52e3d23012ac4bec811768c79781b476" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 43757 } } }
I20260808 06:09:09.106398 4167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52e3d23012ac4bec811768c79781b476 [sys.catalog]: This master's current role is: LEADER
I20260808 06:09:09.106621 4173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52e3d23012ac4bec811768c79781b476 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52e3d23012ac4bec811768c79781b476" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52e3d23012ac4bec811768c79781b476" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 43757 } } }
I20260808 06:09:09.106734 4173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52e3d23012ac4bec811768c79781b476 [sys.catalog]: This master's current role is: LEADER
I20260808 06:09:09.107069 4183 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:09:09.110303 4183 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:09:09.112882 4159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:09.104011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44257 (local address 127.1.151.190:43757)
0808 06:09:09.104144 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:09.104148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:09.104348 (+ 200us) server_negotiation.cc:407] Connection header received
0808 06:09:09.104469 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:09.104472 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:09.104520 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:09.104599 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:09.105466 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:09.106282 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:09.107381 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:09.107710 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:09.107815 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:09.109324 (+ 1509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:09.109328 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:09.109330 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:09:09.109356 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:09.111048 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:09.111548 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:09.111552 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:09.111554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:09.111651 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:09.111955 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:09.111958 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:09.111960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:09.112182 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:09.112315 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:09.112606 (+ 291us) server_negotiation.cc:299] Negotiation successful
0808 06:09:09.112736 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 06:09:09.117933 4183 catalog_manager.cc:1347] Generated new cluster ID: bab8476e4bb34903b9aab138b062736f
I20260808 06:09:09.117996 4183 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:09:09.130944 4183 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:09:09.131820 4183 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:09:09.135545 4183 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52e3d23012ac4bec811768c79781b476: Generated new TSK 0
I20260808 06:09:09.136180 4183 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 06:09:09.140071 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 4048
2026-08-08T06:09:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1499 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:09:09 dist-test-slave-zj8q krb5kdc[4212](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:09:09 dist-test-slave-zj8q krb5kdc[4212](info): set up 2 sockets
Aug 08 06:09:09 dist-test-slave-zj8q krb5kdc[4212](info): commencing operation
krb5kdc: starting...
W20260808 06:09:10.304893 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:09:10 dist-test-slave-zj8q krb5kdc[4212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:09:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:09:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.151.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
I20260808 06:09:10.425071 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:40159
--webserver_interface=127.1.151.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:38663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:40159
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:09:10.534525 4233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:09:10.534765 4233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:09:10.534807 4233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:09:10.538657 4233 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:09:10.538707 4233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:09:10.538724 4233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:09:10.538738 4233 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:09:10.538753 4233 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:09:10.543812 4233 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:38663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:40159
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:40159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:09:10.544754 4233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:09:10.545719 4233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:09:10.553925 4245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:09:10.553893 4241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:09:10.553876 4242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:09:10.553953 4233 server_base.cc:1034] running on GCE node
I20260808 06:09:10.554782 4233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:09:10.555840 4233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:09:10.557006 4233 hybrid_clock.cc:648] HybridClock initialized: now 1786169350556991 us; error 27 us; skew 500 ppm
Aug 08 06:09:10 dist-test-slave-zj8q krb5kdc[4212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169350, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:09:10.559715 4233 init.cc:377] Logged in from keytab as kudu/127.1.151.190@KRBTEST.COM (short username kudu)
I20260808 06:09:10.560571 4233 webserver.cc:460] Webserver started at http://127.1.151.190:33485/ using document root <none> and password file <none>
I20260808 06:09:10.561106 4233 fs_manager.cc:359] Metadata directory not provided
I20260808 06:09:10.561170 4233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:09:10.561367 4233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:09:10.563486 4233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "e2c4f3c1d7d8465cb58ee8e4050c370c"
format_stamp: "Formatted at 2026-08-08 06:09:10 on dist-test-slave-zj8q"
server_key: "3408d3e714e56755490c0b42a9568a00"
server_key_iv: "571641d6c548833c6ce6df5009ac0702"
server_key_version: "encryptionkey@0"
I20260808 06:09:10.564059 4233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "e2c4f3c1d7d8465cb58ee8e4050c370c"
format_stamp: "Formatted at 2026-08-08 06:09:10 on dist-test-slave-zj8q"
server_key: "3408d3e714e56755490c0b42a9568a00"
server_key_iv: "571641d6c548833c6ce6df5009ac0702"
server_key_version: "encryptionkey@0"
I20260808 06:09:10.567698 4233 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 06:09:10.569947 4256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:09:10.571141 4233 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:09:10.571254 4233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "e2c4f3c1d7d8465cb58ee8e4050c370c"
format_stamp: "Formatted at 2026-08-08 06:09:10 on dist-test-slave-zj8q"
server_key: "3408d3e714e56755490c0b42a9568a00"
server_key_iv: "571641d6c548833c6ce6df5009ac0702"
server_key_version: "encryptionkey@0"
I20260808 06:09:10.571336 4233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:09:10.593003 4233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:09:10.596065 4233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:09:10.596225 4233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:09:10.603214 4233 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:40159
I20260808 06:09:10.603252 4343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:40159 every 8 connection(s)
I20260808 06:09:10.604108 4233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:09:10.606829 4344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:09:10.611255 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 4233
I20260808 06:09:10.611374 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:09:10.611660 1630 external_mini_cluster.cc:1428] Setting key 1e22f9cd3ecf4d7f63262168837ca02a
I20260808 06:09:10.612330 4344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2c4f3c1d7d8465cb58ee8e4050c370c: Bootstrap starting.
I20260808 06:09:10.615054 4344 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2c4f3c1d7d8465cb58ee8e4050c370c: Neither blocks nor log segments found. Creating new log.
I20260808 06:09:10.615921 4344 log.cc:824] T 00000000000000000000000000000000 P e2c4f3c1d7d8465cb58ee8e4050c370c: Log is configured to *not* fsync() on all Append() calls
I20260808 06:09:10.617800 4344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2c4f3c1d7d8465cb58ee8e4050c370c: No bootstrap required, opened a new log
I20260808 06:09:10.621060 4347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:10.612872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44005 (local address 127.1.151.190:40159)
0808 06:09:10.613273 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:10.613292 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:10.613333 (+ 41us) server_negotiation.cc:407] Connection header received
0808 06:09:10.614133 (+ 800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:10.614181 (+ 48us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:10.614594 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:10.614936 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:10.615374 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:10.616675 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:10.617587 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:10.618366 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:10.618535 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:10.619183 (+ 648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:10.619190 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:10.619213 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:09:10.619681 (+ 468us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:10.619748 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:09:10.619777 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:10.619871 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:10.619940 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 06:09:10.620132 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":97,"threads_started":1}
I20260808 06:09:10.621922 4344 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2c4f3c1d7d8465cb58ee8e4050c370c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2c4f3c1d7d8465cb58ee8e4050c370c" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 40159 } }
I20260808 06:09:10.622129 4344 raft_consensus.cc:374] T 00000000000000000000000000000000 P e2c4f3c1d7d8465cb58ee8e4050c370c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:09:10.622208 4344 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2c4f3c1d7d8465cb58ee8e4050c370c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2c4f3c1d7d8465cb58ee8e4050c370c, State: Initialized, Role: FOLLOWER
I20260808 06:09:10.622599 4344 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2c4f3c1d7d8465cb58ee8e4050c370c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2c4f3c1d7d8465cb58ee8e4050c370c" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 40159 } }
I20260808 06:09:10.622735 4344 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2c4f3c1d7d8465cb58ee8e4050c370c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:09:10.622841 4344 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2c4f3c1d7d8465cb58ee8e4050c370c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:09:10.622952 4344 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2c4f3c1d7d8465cb58ee8e4050c370c [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:09:10.623806 1630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 06:09:10.623854 4344 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2c4f3c1d7d8465cb58ee8e4050c370c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2c4f3c1d7d8465cb58ee8e4050c370c" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 40159 } }
I20260808 06:09:10.624168 4344 leader_election.cc:302] T 00000000000000000000000000000000 P e2c4f3c1d7d8465cb58ee8e4050c370c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2c4f3c1d7d8465cb58ee8e4050c370c; no voters:
I20260808 06:09:10.624444 4344 leader_election.cc:288] T 00000000000000000000000000000000 P e2c4f3c1d7d8465cb58ee8e4050c370c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:09:10.624526 4349 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2c4f3c1d7d8465cb58ee8e4050c370c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:09:10.624699 4349 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2c4f3c1d7d8465cb58ee8e4050c370c [term 1 LEADER]: Becoming Leader. State: Replica: e2c4f3c1d7d8465cb58ee8e4050c370c, State: Running, Role: LEADER
I20260808 06:09:10.625049 4349 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2c4f3c1d7d8465cb58ee8e4050c370c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2c4f3c1d7d8465cb58ee8e4050c370c" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 40159 } }
I20260808 06:09:10.625689 4344 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2c4f3c1d7d8465cb58ee8e4050c370c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:09:10.626809 4353 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2c4f3c1d7d8465cb58ee8e4050c370c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e2c4f3c1d7d8465cb58ee8e4050c370c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2c4f3c1d7d8465cb58ee8e4050c370c" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 40159 } } }
I20260808 06:09:10.626955 4353 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2c4f3c1d7d8465cb58ee8e4050c370c [sys.catalog]: This master's current role is: LEADER
I20260808 06:09:10.627283 4354 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2c4f3c1d7d8465cb58ee8e4050c370c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2c4f3c1d7d8465cb58ee8e4050c370c. Latest consensus state: current_term: 1 leader_uuid: "e2c4f3c1d7d8465cb58ee8e4050c370c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2c4f3c1d7d8465cb58ee8e4050c370c" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 40159 } } }
I20260808 06:09:10.627379 4354 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2c4f3c1d7d8465cb58ee8e4050c370c [sys.catalog]: This master's current role is: LEADER
I20260808 06:09:10.628257 4372 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:09:10.630515 4347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:10.625779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42669 (local address 127.1.151.190:40159)
0808 06:09:10.625882 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:10.625886 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:10.626417 (+ 531us) server_negotiation.cc:407] Connection header received
0808 06:09:10.626632 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:10.626635 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:10.626680 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:10.626728 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:10.627141 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:10.627834 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:10.628962 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:10.629290 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:10.629399 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:10.630088 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:10.630092 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:10.630095 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:09:10.630108 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:10.630137 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:09:10.630171 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:10.630241 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:10.630336 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 06:09:10.630388 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 06:09:10.631668 4372 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:09:10.636811 4372 catalog_manager.cc:1347] Generated new cluster ID: 578c5162e61249e79058edd5a47ceaa6
I20260808 06:09:10.636909 4372 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:09:10.642445 4372 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:09:10.643443 4372 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:09:10.647596 4372 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e2c4f3c1d7d8465cb58ee8e4050c370c: Generated new TSK 0
I20260808 06:09:10.648315 4372 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 06:09:10.652477 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 4233
2026-08-08T06:09:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1519 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:09:10 dist-test-slave-zj8q krb5kdc[4386](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:09:10 dist-test-slave-zj8q krb5kdc[4386](info): set up 2 sockets
Aug 08 06:09:10 dist-test-slave-zj8q krb5kdc[4386](info): commencing operation
krb5kdc: starting...
W20260808 06:09:11.801980 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:09:11 dist-test-slave-zj8q krb5kdc[4386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:09:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:09:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.151.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
I20260808 06:09:11.922648 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:40667
--webserver_interface=127.1.151.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:40517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:40667
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:09:12.033310 4414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:09:12.033569 4414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:09:12.033612 4414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:09:12.037622 4414 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:09:12.037685 4414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:09:12.037704 4414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:09:12.037720 4414 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:09:12.037751 4414 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:09:12.042968 4414 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:40517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:40667
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:40667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:09:12.043967 4414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:09:12.044975 4414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:09:12.053385 4421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:09:12.053419 4420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:09:12.053552 4414 server_base.cc:1034] running on GCE node
W20260808 06:09:12.053392 4424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:09:12.054328 4414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:09:12.055419 4414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:09:12.056572 4414 hybrid_clock.cc:648] HybridClock initialized: now 1786169352056550 us; error 36 us; skew 500 ppm
Aug 08 06:09:12 dist-test-slave-zj8q krb5kdc[4386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169352, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:09:12.059386 4414 init.cc:377] Logged in from keytab as kudu/127.1.151.190@KRBTEST.COM (short username kudu)
I20260808 06:09:12.060276 4414 webserver.cc:460] Webserver started at http://127.1.151.190:38137/ using document root <none> and password file <none>
I20260808 06:09:12.060822 4414 fs_manager.cc:359] Metadata directory not provided
I20260808 06:09:12.060887 4414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:09:12.061074 4414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:09:12.063166 4414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "1e126cf45b6841a392acb6e5e1a11f7f"
format_stamp: "Formatted at 2026-08-08 06:09:12 on dist-test-slave-zj8q"
server_key: "c35214473be89c028a0f24f8b5d212c3"
server_key_iv: "6ca91743ada25575eeb333680719b5ca"
server_key_version: "encryptionkey@0"
I20260808 06:09:12.063710 4414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "1e126cf45b6841a392acb6e5e1a11f7f"
format_stamp: "Formatted at 2026-08-08 06:09:12 on dist-test-slave-zj8q"
server_key: "c35214473be89c028a0f24f8b5d212c3"
server_key_iv: "6ca91743ada25575eeb333680719b5ca"
server_key_version: "encryptionkey@0"
I20260808 06:09:12.067456 4414 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 06:09:12.069778 4436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:09:12.071019 4414 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:09:12.071156 4414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "1e126cf45b6841a392acb6e5e1a11f7f"
format_stamp: "Formatted at 2026-08-08 06:09:12 on dist-test-slave-zj8q"
server_key: "c35214473be89c028a0f24f8b5d212c3"
server_key_iv: "6ca91743ada25575eeb333680719b5ca"
server_key_version: "encryptionkey@0"
I20260808 06:09:12.071251 4414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:09:12.075619 4414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:09:12.078418 4414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:09:12.078594 4414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:09:12.085455 4414 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:40667
I20260808 06:09:12.085466 4511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:40667 every 8 connection(s)
I20260808 06:09:12.086386 4414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:09:12.088411 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 4414
I20260808 06:09:12.088548 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:09:12.088857 1630 external_mini_cluster.cc:1428] Setting key e9783e6d11c2b628a0250ed29ff838e9
I20260808 06:09:12.089263 4512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:09:12.093849 4518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:12.090278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53205 (local address 127.1.151.190:40667)
0808 06:09:12.090753 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:12.090768 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:12.090803 (+ 35us) server_negotiation.cc:407] Connection header received
0808 06:09:12.091440 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:12.091457 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:12.091726 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:12.092016 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:12.092746 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:12.092752 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:12.092764 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:09:12.092774 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:12.092826 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:09:12.092846 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:12.092967 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:12.093043 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 06:09:12.093191 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":147,"threads_started":1}
I20260808 06:09:12.095434 4512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e126cf45b6841a392acb6e5e1a11f7f: Bootstrap starting.
I20260808 06:09:12.096545 1630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 06:09:12.097882 4512 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e126cf45b6841a392acb6e5e1a11f7f: Neither blocks nor log segments found. Creating new log.
I20260808 06:09:12.098810 4512 log.cc:824] T 00000000000000000000000000000000 P 1e126cf45b6841a392acb6e5e1a11f7f: Log is configured to *not* fsync() on all Append() calls
I20260808 06:09:12.100992 4512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e126cf45b6841a392acb6e5e1a11f7f: No bootstrap required, opened a new log
I20260808 06:09:12.103636 4518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:12.099005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39019 (local address 127.1.151.190:40667)
0808 06:09:12.099152 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:12.099156 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:12.102422 (+ 3266us) server_negotiation.cc:407] Connection header received
0808 06:09:12.102536 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:12.102538 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:12.102569 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:12.102633 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:12.103299 (+ 666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:12.103302 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:12.103304 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 06:09:12.103314 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:12.103339 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 06:09:12.103349 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:12.103414 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:12.103500 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 06:09:12.103530 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 06:09:12.104957 4512 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e126cf45b6841a392acb6e5e1a11f7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e126cf45b6841a392acb6e5e1a11f7f" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 40667 } }
I20260808 06:09:12.105172 4512 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e126cf45b6841a392acb6e5e1a11f7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 06:09:12.105229 4512 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e126cf45b6841a392acb6e5e1a11f7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e126cf45b6841a392acb6e5e1a11f7f, State: Initialized, Role: FOLLOWER
I20260808 06:09:12.105644 4512 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e126cf45b6841a392acb6e5e1a11f7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e126cf45b6841a392acb6e5e1a11f7f" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 40667 } }
I20260808 06:09:12.105773 4512 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e126cf45b6841a392acb6e5e1a11f7f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 06:09:12.105835 4512 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e126cf45b6841a392acb6e5e1a11f7f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 06:09:12.105927 4512 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e126cf45b6841a392acb6e5e1a11f7f [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:09:12.106778 4512 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e126cf45b6841a392acb6e5e1a11f7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e126cf45b6841a392acb6e5e1a11f7f" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 40667 } }
I20260808 06:09:12.107082 4512 leader_election.cc:302] T 00000000000000000000000000000000 P 1e126cf45b6841a392acb6e5e1a11f7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e126cf45b6841a392acb6e5e1a11f7f; no voters:
I20260808 06:09:12.107359 4512 leader_election.cc:288] T 00000000000000000000000000000000 P 1e126cf45b6841a392acb6e5e1a11f7f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 06:09:12.107451 4527 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e126cf45b6841a392acb6e5e1a11f7f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:09:12.107668 4527 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e126cf45b6841a392acb6e5e1a11f7f [term 1 LEADER]: Becoming Leader. State: Replica: 1e126cf45b6841a392acb6e5e1a11f7f, State: Running, Role: LEADER
I20260808 06:09:12.108034 4527 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e126cf45b6841a392acb6e5e1a11f7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e126cf45b6841a392acb6e5e1a11f7f" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 40667 } }
I20260808 06:09:12.108476 4512 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e126cf45b6841a392acb6e5e1a11f7f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:09:12.109563 4528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e126cf45b6841a392acb6e5e1a11f7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e126cf45b6841a392acb6e5e1a11f7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e126cf45b6841a392acb6e5e1a11f7f" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 40667 } } }
I20260808 06:09:12.109683 4528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e126cf45b6841a392acb6e5e1a11f7f [sys.catalog]: This master's current role is: LEADER
I20260808 06:09:12.110097 4542 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:09:12.110486 4529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e126cf45b6841a392acb6e5e1a11f7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e126cf45b6841a392acb6e5e1a11f7f. Latest consensus state: current_term: 1 leader_uuid: "1e126cf45b6841a392acb6e5e1a11f7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e126cf45b6841a392acb6e5e1a11f7f" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 40667 } } }
I20260808 06:09:12.110647 4529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e126cf45b6841a392acb6e5e1a11f7f [sys.catalog]: This master's current role is: LEADER
I20260808 06:09:12.112747 4542 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:09:12.118294 4542 catalog_manager.cc:1347] Generated new cluster ID: 86256e9b7c9c43c684c5055b9e7c2275
I20260808 06:09:12.118357 4542 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:09:12.136814 4542 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:09:12.137110 4542 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:09:12.145316 4542 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e126cf45b6841a392acb6e5e1a11f7f: Generated new TSK 0
I20260808 06:09:12.145967 4542 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 06:09:12.149530 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 4414
2026-08-08T06:09:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1487 ms)
[----------] 18 tests from AuthTokenIssuingTest (27107 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:09:12 dist-test-slave-zj8q krb5kdc[4557](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:09:12 dist-test-slave-zj8q krb5kdc[4557](info): set up 2 sockets
Aug 08 06:09:12 dist-test-slave-zj8q krb5kdc[4557](info): commencing operation
krb5kdc: starting...
W20260808 06:09:13.304551 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:09:13 dist-test-slave-zj8q krb5kdc[4557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:09:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:09:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.151.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
I20260808 06:09:13.425135 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:44347
--webserver_interface=127.1.151.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:42519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:44347,127.1.151.189:43243,127.1.151.188:44323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:09:13.535271 4573 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 06:09:13.535595 4573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:09:13.535688 4573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:09:13.535754 4573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:09:13.539902 4573 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:09:13.539952 4573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:09:13.539970 4573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:09:13.539986 4573 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:09:13.539999 4573 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:09:13.545020 4573 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:42519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:44347,127.1.151.189:43243,127.1.151.188:44323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:44347
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:09:13.546021 4573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:09:13.547053 4573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:09:13.555404 4582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:09:13.555460 4573 server_base.cc:1034] running on GCE node
W20260808 06:09:13.555415 4583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:09:13.555405 4588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:09:13.556356 4573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:09:13.557452 4573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:09:13.558632 4573 hybrid_clock.cc:648] HybridClock initialized: now 1786169353558605 us; error 35 us; skew 500 ppm
Aug 08 06:09:13 dist-test-slave-zj8q krb5kdc[4557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169353, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:09:13.561631 4573 init.cc:377] Logged in from keytab as kudu/127.1.151.190@KRBTEST.COM (short username kudu)
I20260808 06:09:13.562615 4573 webserver.cc:460] Webserver started at http://127.1.151.190:36497/ using document root <none> and password file <none>
I20260808 06:09:13.563179 4573 fs_manager.cc:359] Metadata directory not provided
I20260808 06:09:13.563249 4573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:09:13.563455 4573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:09:13.565500 4573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "c47b9213bb0b4faf97471d591b1dbf49"
format_stamp: "Formatted at 2026-08-08 06:09:13 on dist-test-slave-zj8q"
server_key: "d6c188515d4c9e7b18357d4bd2e3dfdf"
server_key_iv: "97883a0789e03c2c38f74cc38d1eb2f4"
server_key_version: "encryptionkey@0"
I20260808 06:09:13.566073 4573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "c47b9213bb0b4faf97471d591b1dbf49"
format_stamp: "Formatted at 2026-08-08 06:09:13 on dist-test-slave-zj8q"
server_key: "d6c188515d4c9e7b18357d4bd2e3dfdf"
server_key_iv: "97883a0789e03c2c38f74cc38d1eb2f4"
server_key_version: "encryptionkey@0"
I20260808 06:09:13.569720 4573 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 06:09:13.572206 4597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:09:13.573378 4573 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:09:13.573509 4573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "c47b9213bb0b4faf97471d591b1dbf49"
format_stamp: "Formatted at 2026-08-08 06:09:13 on dist-test-slave-zj8q"
server_key: "d6c188515d4c9e7b18357d4bd2e3dfdf"
server_key_iv: "97883a0789e03c2c38f74cc38d1eb2f4"
server_key_version: "encryptionkey@0"
I20260808 06:09:13.573606 4573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:09:13.592407 4573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:09:13.595396 4573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:09:13.595579 4573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:09:13.602351 4573 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:44347
I20260808 06:09:13.602605 4679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:44347 every 8 connection(s)
I20260808 06:09:13.603276 4573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:09:13.606112 4681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:09:13.610198 4681 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44347 } has no permanent_uuid. Determining permanent_uuid...
I20260808 06:09:13.611168 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 4573
I20260808 06:09:13.611292 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:09:13.611593 1630 external_mini_cluster.cc:1428] Setting key fceba27b7766b451321f5761f8c9f5f5
WARNING: no policy specified for kudu/127.1.151.189@KRBTEST.COM; defaulting to no policy
Aug 08 06:09:13 dist-test-slave-zj8q krb5kdc[4557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169353, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.189@KRBTEST.COM" created.
I20260808 06:09:13.624950 4685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:13.612726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:44347 (local address 127.0.0.1:38834)
0808 06:09:13.613359 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:13.613406 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 06:09:13.614256 (+ 850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:09:13.614915 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:09:13.614920 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:09:13.615343 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:09:13.615576 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:09:13.615589 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:09:13.616701 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:09:13.616703 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:09:13.617413 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:09:13.617422 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:09:13.618022 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:09:13.618026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:09:13.618237 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:13.618785 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:09:13.618800 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:09:13.620335 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:09:13.622206 (+ 1871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:09:13.622215 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:09:13.622223 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:09:13.622440 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:09:13.622837 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:09:13.622840 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:09:13.622841 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:09:13.622954 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:09:13.623540 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:09:13.623546 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:09:13.623675 (+ 129us) client_negotiation.cc:769] Sending connection context
0808 06:09:13.623810 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 06:09:13.624221 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"mutex_wait_us":167,"spinlock_wait_cycles":44416,"thread_start_us":149,"threads_started":1}
I20260808 06:09:13.624956 4684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:13.612726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38834 (local address 127.1.151.190:44347)
0808 06:09:13.613188 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:13.613206 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:13.613584 (+ 378us) server_negotiation.cc:407] Connection header received
0808 06:09:13.614552 (+ 968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:13.614556 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:13.614735 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:13.614894 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:13.615719 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:13.616582 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:13.617525 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:13.617880 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:13.618064 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:13.620481 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:13.620487 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:13.620496 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:09:13.620519 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:13.622050 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:13.622547 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:13.622554 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:13.622556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:13.622747 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:13.623059 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:13.623063 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:13.623064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:13.623430 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:13.623551 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:13.623909 (+ 358us) server_negotiation.cc:299] Negotiation successful
0808 06:09:13.624110 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":307,"thread_start_us":145,"threads_started":1}
I20260808 06:09:13.627257 4681 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.151.189" port: 43243 } has no permanent_uuid. Determining permanent_uuid...
I20260808 06:09:13.627918 4685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:13.627692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.189:43243 (local address 127.0.0.1:42398)
0808 06:09:13.627803 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:13.627867 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.189:43243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 06:09:13.628079 4599 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.151.189:43243: connect: Connection refused (error 111)
I20260808 06:09:13.628813 4685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:13.628632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.189:43243 (local address 127.0.0.1:42414)
0808 06:09:13.628726 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:13.628767 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.189:43243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 06:09:13.628932 4681 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.151.189:43243: Network error: Client connection negotiation failed: client connection to 127.1.151.189:43243: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.189.keytab.
Entry for principal kudu/127.1.151.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.189.keytab.
I20260808 06:09:13.638424 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.189:43243
--webserver_interface=127.1.151.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.189.keytab
--principal=kudu/127.1.151.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:42519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:44347,127.1.151.189:43243,127.1.151.188:44323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 06:09:13.678171 4681 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.151.189" port: 43243 } attempt: 1
I20260808 06:09:13.679015 4685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:13.678728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.189:43243 (local address 127.0.0.1:42430)
0808 06:09:13.678876 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:13.678938 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.189:43243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 06:09:13.679708 4685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:13.679482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.189:43243 (local address 127.0.0.1:42442)
0808 06:09:13.679589 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:13.679643 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.189:43243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 06:09:13.679860 4681 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.151.189:43243: Network error: Client connection negotiation failed: client connection to 127.1.151.189:43243: connect: Connection refused (error 111)
I20260808 06:09:13.748019 4681 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.151.189" port: 43243 } attempt: 2
I20260808 06:09:13.748904 4685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:13.748593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.189:43243 (local address 127.0.0.1:42458)
0808 06:09:13.748759 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:13.748824 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.189:43243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260808 06:09:13.749176 4694 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 06:09:13.749431 4694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:09:13.749475 4694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:09:13.749564 4694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 06:09:13.749658 4685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:13.749403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.189:43243 (local address 127.0.0.1:42464)
0808 06:09:13.749539 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:13.749583 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.189:43243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260808 06:09:13.749799 4681 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.151.189:43243: Network error: Client connection negotiation failed: client connection to 127.1.151.189:43243: connect: Connection refused (error 111)
W20260808 06:09:13.753641 4694 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:09:13.753693 4694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:09:13.753711 4694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:09:13.753726 4694 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:09:13.753747 4694 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:09:13.759123 4694 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:42519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:44347,127.1.151.189:43243,127.1.151.188:44323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.189.keytab
--principal=kudu/127.1.151.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.189:43243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.151.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:09:13.760102 4694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:09:13.761046 4694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:09:13.768982 4703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:09:13.769014 4701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:09:13.769104 4700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:09:13.769476 4694 server_base.cc:1034] running on GCE node
I20260808 06:09:13.770068 4694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:09:13.771085 4694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:09:13.772248 4694 hybrid_clock.cc:648] HybridClock initialized: now 1786169353772220 us; error 36 us; skew 500 ppm
Aug 08 06:09:13 dist-test-slave-zj8q krb5kdc[4557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169353, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:09:13.774981 4694 init.cc:377] Logged in from keytab as kudu/127.1.151.189@KRBTEST.COM (short username kudu)
I20260808 06:09:13.775763 4694 webserver.cc:460] Webserver started at http://127.1.151.189:42561/ using document root <none> and password file <none>
I20260808 06:09:13.776264 4694 fs_manager.cc:359] Metadata directory not provided
I20260808 06:09:13.776328 4694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:09:13.776510 4694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:09:13.778515 4694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-1/data/instance:
uuid: "37b4fae21b784ecf850f14670cf81e44"
format_stamp: "Formatted at 2026-08-08 06:09:13 on dist-test-slave-zj8q"
server_key: "c024d4892524bbdfa4dc688aab43650f"
server_key_iv: "0dd426b5de8b95072bc911a51af18f5d"
server_key_version: "encryptionkey@0"
I20260808 06:09:13.779035 4694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-1/wal/instance:
uuid: "37b4fae21b784ecf850f14670cf81e44"
format_stamp: "Formatted at 2026-08-08 06:09:13 on dist-test-slave-zj8q"
server_key: "c024d4892524bbdfa4dc688aab43650f"
server_key_iv: "0dd426b5de8b95072bc911a51af18f5d"
server_key_version: "encryptionkey@0"
I20260808 06:09:13.782420 4694 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 06:09:13.784770 4721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:09:13.785923 4694 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 06:09:13.786042 4694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-1/wal
uuid: "37b4fae21b784ecf850f14670cf81e44"
format_stamp: "Formatted at 2026-08-08 06:09:13 on dist-test-slave-zj8q"
server_key: "c024d4892524bbdfa4dc688aab43650f"
server_key_iv: "0dd426b5de8b95072bc911a51af18f5d"
server_key_version: "encryptionkey@0"
I20260808 06:09:13.786136 4694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:09:13.798566 4694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:09:13.801350 4694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:09:13.801527 4694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:09:13.808543 4694 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.189:43243
I20260808 06:09:13.808841 4800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.189:43243 every 8 connection(s)
I20260808 06:09:13.809494 4694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-1/data/info.pb
I20260808 06:09:13.812270 4801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:09:13.814268 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 4694
I20260808 06:09:13.814386 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-1/wal/instance
I20260808 06:09:13.814697 1630 external_mini_cluster.cc:1428] Setting key ea0efea30f0e91f58ef642a081694f25
I20260808 06:09:13.816780 4801 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44347 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.151.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.188@KRBTEST.COM" created.
Aug 08 06:09:13 dist-test-slave-zj8q krb5kdc[4557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169353, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.189@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:09:13.830840 4684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:13.818733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38838 (local address 127.1.151.190:44347)
0808 06:09:13.818860 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:13.818864 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:13.819909 (+ 1045us) server_negotiation.cc:407] Connection header received
0808 06:09:13.820886 (+ 977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:13.820890 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:13.820951 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:13.821026 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:13.822231 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:13.822778 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:13.823981 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:13.824225 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:13.824319 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:13.827327 (+ 3008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:13.827331 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:13.827334 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:09:13.827365 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:13.828706 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:13.829295 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:13.829297 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:13.829299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:13.829372 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:13.829720 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:13.829722 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:13.829723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:13.829889 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:13.829959 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:13.830614 (+ 655us) server_negotiation.cc:299] Negotiation successful
0808 06:09:13.830715 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 06:09:13.831962 4807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:13.819241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:44347 (local address 127.0.0.1:38838)
0808 06:09:13.819747 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:13.819790 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 06:09:13.820636 (+ 846us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:09:13.821229 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:09:13.821236 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:09:13.821580 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:09:13.822060 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:09:13.822074 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:09:13.822904 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:09:13.822907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:09:13.823863 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:09:13.823873 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:09:13.824373 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:09:13.824377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:09:13.824588 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:13.825164 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:09:13.825181 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:09:13.827147 (+ 1966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:09:13.828828 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:09:13.828834 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:09:13.828849 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:09:13.829204 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:09:13.829452 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:09:13.829455 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:09:13.829458 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:09:13.829637 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:09:13.829989 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:09:13.829995 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:09:13.830368 (+ 373us) client_negotiation.cc:769] Sending connection context
0808 06:09:13.830570 (+ 202us) client_negotiation.cc:240] Negotiation successful
0808 06:09:13.830938 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":137,"threads_started":1}
I20260808 06:09:13.833400 4801 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.151.189" port: 43243 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 06:09:13 dist-test-slave-zj8q krb5kdc[4557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169353, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.189@KRBTEST.COM for kudu/127.1.151.189@KRBTEST.COM
I20260808 06:09:13.840966 4681 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.151.189" port: 43243 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.188.keytab.
Entry for principal kudu/127.1.151.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.188.keytab.
I20260808 06:09:13.842736 4807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:13.833908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.189:43243 (local address 127.0.0.1:42470)
0808 06:09:13.834004 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:13.834018 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:09:13.834082 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:09:13.834684 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:09:13.834688 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:09:13.834851 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:09:13.834976 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:09:13.834984 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:09:13.835886 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:09:13.835889 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:09:13.836456 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:09:13.836465 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:09:13.837007 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:09:13.837010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:09:13.837086 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:13.837476 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:09:13.837490 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:09:13.838960 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:09:13.840973 (+ 2013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:09:13.840977 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:09:13.840978 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:09:13.841224 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:09:13.841552 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:09:13.841555 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:09:13.841557 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:09:13.841656 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:09:13.842171 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:09:13.842175 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:09:13.842272 (+ 97us) client_negotiation.cc:769] Sending connection context
0808 06:09:13.842392 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 06:09:13.842519 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 06:09:13.842864 4810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:13.834027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42470 (local address 127.1.151.189:43243)
0808 06:09:13.834352 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:13.834360 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:13.834378 (+ 18us) server_negotiation.cc:407] Connection header received
0808 06:09:13.834409 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:13.834413 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:13.834552 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:13.834655 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:13.835083 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:13.835784 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:13.836550 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:13.836901 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:13.836978 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:13.839102 (+ 2124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:13.839111 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:13.839122 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:09:13.839159 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:13.840860 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:13.841342 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:13.841349 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:13.841350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:13.841444 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:13.841757 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:13.841759 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:13.841762 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:13.842048 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:13.842195 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:13.842484 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 06:09:13.842680 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":154,"threads_started":1}
I20260808 06:09:13.843334 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.188:44323
--webserver_interface=127.1.151.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.188.keytab
--principal=kudu/127.1.151.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:42519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:44347,127.1.151.189:43243,127.1.151.188:44323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 06:09:13.844672 4801 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44323 } has no permanent_uuid. Determining permanent_uuid...
I20260808 06:09:13.845371 4807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:13.845146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:44323 (local address 127.0.0.1:41760)
0808 06:09:13.845233 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:13.845307 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 06:09:13.846081 4722 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44323: connect: Connection refused (error 111)
Aug 08 06:09:13 dist-test-slave-zj8q krb5kdc[4557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169353, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for kudu/127.1.151.189@KRBTEST.COM
I20260808 06:09:13.846966 4807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:13.846745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:44323 (local address 127.0.0.1:41774)
0808 06:09:13.846857 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:13.846909 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 06:09:13.847126 4801 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.151.188:44323: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44323: connect: Connection refused (error 111)
I20260808 06:09:13.849833 4685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:13.841443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.189:43243 (local address 127.0.0.1:42482)
0808 06:09:13.841597 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:13.841613 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 06:09:13.841691 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:09:13.842044 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:09:13.842047 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:09:13.842253 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:09:13.842365 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:09:13.842373 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:09:13.843298 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:09:13.843302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:09:13.844539 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:09:13.844552 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:09:13.845060 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:09:13.845064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:09:13.845174 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:13.845603 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:09:13.845615 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:09:13.846874 (+ 1259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:09:13.848509 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:09:13.848512 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:09:13.848514 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:09:13.848703 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:09:13.848945 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:09:13.848948 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:09:13.848950 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:09:13.849028 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:09:13.849363 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:09:13.849366 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:09:13.849457 (+ 91us) client_negotiation.cc:769] Sending connection context
0808 06:09:13.849541 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 06:09:13.849674 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 06:09:13.849857 4812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:13.841491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42482 (local address 127.1.151.189:43243)
0808 06:09:13.841849 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:13.841858 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:13.841871 (+ 13us) server_negotiation.cc:407] Connection header received
0808 06:09:13.841901 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:13.841904 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:13.841942 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:13.842022 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:13.842484 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:13.843184 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:13.844657 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:13.844961 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:13.845065 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:13.847030 (+ 1965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:13.847034 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:13.847037 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:09:13.847062 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:13.848411 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:13.848788 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:13.848791 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:13.848793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:13.848866 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:13.849112 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:13.849115 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:13.849117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:13.849277 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:13.849358 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:13.849609 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 06:09:13.849705 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":113,"threads_started":1}
I20260808 06:09:13.850530 4681 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44323 } has no permanent_uuid. Determining permanent_uuid...
I20260808 06:09:13.851195 4685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:13.850944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:44323 (local address 127.0.0.1:41788)
0808 06:09:13.851063 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:13.851118 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 06:09:13.851881 4685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:13.851605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:44323 (local address 127.0.0.1:41790)
0808 06:09:13.851761 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:13.851813 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260808 06:09:13.852020 4681 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.151.188:44323: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44323: connect: Connection refused (error 111)
I20260808 06:09:13.883177 4681 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44323 } attempt: 1
I20260808 06:09:13.883936 4685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:13.883611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:44323 (local address 127.0.0.1:41794)
0808 06:09:13.883765 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:13.883845 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 06:09:13.884634 4685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:13.884407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:44323 (local address 127.0.0.1:41808)
0808 06:09:13.884504 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:13.884555 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 06:09:13.884761 4681 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.151.188:44323: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44323: connect: Connection refused (error 111)
I20260808 06:09:13.896356 4801 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44323 } attempt: 1
I20260808 06:09:13.897037 4807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:13.896787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:44323 (local address 127.0.0.1:41810)
0808 06:09:13.896915 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:13.896969 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 06:09:13.897665 4807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:13.897477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:44323 (local address 127.0.0.1:41814)
0808 06:09:13.897565 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:13.897606 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 06:09:13.897785 4801 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.151.188:44323: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44323: connect: Connection refused (error 111)
W20260808 06:09:13.953338 4813 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 06:09:13.953585 4813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:09:13.953606 4813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:09:13.953648 4813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:09:13.957469 4813 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:09:13.957515 4813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:09:13.957532 4813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:09:13.957547 4813 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:09:13.957568 4813 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:09:13.959931 4681 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44323 } attempt: 2
I20260808 06:09:13.960759 4685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:13.960437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:44323 (local address 127.0.0.1:41826)
0808 06:09:13.960596 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:13.960670 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 06:09:13.961431 4685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:13.961224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:44323 (local address 127.0.0.1:41838)
0808 06:09:13.961317 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:13.961366 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 06:09:13.961539 4681 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.151.188:44323: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44323: connect: Connection refused (error 111)
I20260808 06:09:13.962980 4813 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:42519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:44347,127.1.151.189:43243,127.1.151.188:44323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.188.keytab
--principal=kudu/127.1.151.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.188:44323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.151.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:09:13.963915 4813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:09:13.964877 4813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 06:09:13.965929 4801 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44323 } attempt: 2
I20260808 06:09:13.966578 4807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:13.966344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:44323 (local address 127.0.0.1:41842)
0808 06:09:13.966464 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:13.966513 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 06:09:13.967231 4807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:13.967030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:44323 (local address 127.0.0.1:41854)
0808 06:09:13.967130 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:13.967179 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 06:09:13.967345 4801 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.151.188:44323: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44323: connect: Connection refused (error 111)
W20260808 06:09:13.973240 4828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:09:13.973306 4824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:09:13.973273 4826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:09:13.973389 4813 server_base.cc:1034] running on GCE node
I20260808 06:09:13.973987 4813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:09:13.975023 4813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:09:13.976177 4813 hybrid_clock.cc:648] HybridClock initialized: now 1786169353976159 us; error 23 us; skew 500 ppm
Aug 08 06:09:13 dist-test-slave-zj8q krb5kdc[4557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169353, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:09:13.979146 4813 init.cc:377] Logged in from keytab as kudu/127.1.151.188@KRBTEST.COM (short username kudu)
I20260808 06:09:13.980005 4813 webserver.cc:460] Webserver started at http://127.1.151.188:41523/ using document root <none> and password file <none>
I20260808 06:09:13.980515 4813 fs_manager.cc:359] Metadata directory not provided
I20260808 06:09:13.980578 4813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:09:13.980772 4813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:09:13.982816 4813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-2/data/instance:
uuid: "7ba0e7f5b9784b6ea29877a31ddbbe26"
format_stamp: "Formatted at 2026-08-08 06:09:13 on dist-test-slave-zj8q"
server_key: "a4f6279e46396f8c8cfa2814c11cecd6"
server_key_iv: "b2a87fbf6b6cb5d2a9dafe1f8f587073"
server_key_version: "encryptionkey@0"
I20260808 06:09:13.983345 4813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-2/wal/instance:
uuid: "7ba0e7f5b9784b6ea29877a31ddbbe26"
format_stamp: "Formatted at 2026-08-08 06:09:13 on dist-test-slave-zj8q"
server_key: "a4f6279e46396f8c8cfa2814c11cecd6"
server_key_iv: "b2a87fbf6b6cb5d2a9dafe1f8f587073"
server_key_version: "encryptionkey@0"
I20260808 06:09:13.986879 4813 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 06:09:13.989179 4840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:09:13.990447 4813 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:09:13.990561 4813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-2/wal
uuid: "7ba0e7f5b9784b6ea29877a31ddbbe26"
format_stamp: "Formatted at 2026-08-08 06:09:13 on dist-test-slave-zj8q"
server_key: "a4f6279e46396f8c8cfa2814c11cecd6"
server_key_iv: "b2a87fbf6b6cb5d2a9dafe1f8f587073"
server_key_version: "encryptionkey@0"
I20260808 06:09:13.990694 4813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:09:14.010641 4813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:09:14.013720 4813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:09:14.013893 4813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:09:14.021431 4813 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.188:44323
I20260808 06:09:14.021710 4925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.188:44323 every 8 connection(s)
I20260808 06:09:14.022464 4813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-2/data/info.pb
I20260808 06:09:14.025096 4926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:09:14.028823 4926 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44347 } has no permanent_uuid. Determining permanent_uuid...
I20260808 06:09:14.030376 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 4813
I20260808 06:09:14.030465 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786169070948234-1630-0/minicluster-data/master-2/wal/instance
I20260808 06:09:14.030768 1630 external_mini_cluster.cc:1428] Setting key 8edc0db46c1345a6a6d0023eeb36c6fc
Aug 08 06:09:14 dist-test-slave-zj8q krb5kdc[4557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169353, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.188@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
Aug 08 06:09:14 dist-test-slave-zj8q krb5kdc[4557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.188@KRBTEST.COM
I20260808 06:09:14.045961 4684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:14.030667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38852 (local address 127.1.151.190:44347)
0808 06:09:14.030819 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:14.030824 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:14.031753 (+ 929us) server_negotiation.cc:407] Connection header received
0808 06:09:14.032805 (+ 1052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:14.032808 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:14.032852 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:14.032911 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:14.033802 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:14.034334 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:14.035396 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:14.035681 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:14.035758 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:14.042210 (+ 6452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:14.042213 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:14.042215 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:09:14.042243 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:14.043670 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:14.044331 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:14.044334 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:14.044336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:14.044417 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:14.044828 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:14.044831 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:14.044833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:14.045066 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:14.045169 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:14.045666 (+ 497us) server_negotiation.cc:299] Negotiation successful
0808 06:09:14.045795 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 06:09:14.046828 4928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:14.031007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:44347 (local address 127.0.0.1:38852)
0808 06:09:14.031541 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:14.031596 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 06:09:14.032589 (+ 993us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:09:14.032980 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:09:14.032985 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:09:14.033406 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:09:14.033656 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:09:14.033675 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:09:14.034431 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:09:14.034434 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:09:14.035273 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:09:14.035282 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:09:14.035814 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:09:14.035819 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:09:14.035992 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:14.040527 (+ 4535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:09:14.040544 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:09:14.041994 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:09:14.043827 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:09:14.043835 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:09:14.043843 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:09:14.044229 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:09:14.044505 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:09:14.044508 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:09:14.044510 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:09:14.044735 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:09:14.045174 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:09:14.045179 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:09:14.045406 (+ 227us) client_negotiation.cc:769] Sending connection context
0808 06:09:14.045591 (+ 185us) client_negotiation.cc:240] Negotiation successful
0808 06:09:14.045908 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":144,"threads_started":1}
I20260808 06:09:14.046828 4930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:14.031412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41870 (local address 127.1.151.188:44323)
0808 06:09:14.031789 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:14.031804 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:14.031930 (+ 126us) server_negotiation.cc:407] Connection header received
0808 06:09:14.032594 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:14.032619 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:14.032852 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:14.033024 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:14.033682 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:14.034740 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:14.035612 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:14.036059 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:14.036195 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:14.042556 (+ 6361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:14.042562 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:14.042571 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:09:14.042592 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:14.044224 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:14.044667 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:14.044672 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:14.044673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:14.044747 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:14.045002 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:14.045005 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:14.045007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:14.045429 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:14.045628 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:14.045841 (+ 213us) server_negotiation.cc:299] Negotiation successful
0808 06:09:14.045975 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":295,"thread_start_us":122,"threads_started":1}
I20260808 06:09:14.048116 4926 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.151.189" port: 43243 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 06:09:14 dist-test-slave-zj8q krb5kdc[4557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169353, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.188@KRBTEST.COM for kudu/127.1.151.189@KRBTEST.COM
Aug 08 06:09:14 dist-test-slave-zj8q krb5kdc[4557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:09:14.057040 4928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:14.048619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.189:43243 (local address 127.0.0.1:42486)
0808 06:09:14.048935 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:14.048949 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:09:14.049021 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:09:14.049443 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:09:14.049446 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:09:14.049607 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:09:14.049722 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:09:14.049729 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:09:14.050738 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:09:14.050741 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:09:14.051470 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:09:14.051480 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:09:14.051990 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:09:14.051993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:09:14.052068 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:14.052458 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:09:14.052472 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:09:14.053768 (+ 1296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:09:14.055588 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:09:14.055591 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:09:14.055592 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:09:14.055797 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:09:14.056085 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:09:14.056087 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:09:14.056089 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:09:14.056176 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:09:14.056533 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:09:14.056536 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:09:14.056616 (+ 80us) client_negotiation.cc:769] Sending connection context
0808 06:09:14.056752 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 06:09:14.056922 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245}
I20260808 06:09:14.057101 4812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:14.048957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42486 (local address 127.1.151.189:43243)
0808 06:09:14.049108 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:14.049112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:14.049127 (+ 15us) server_negotiation.cc:407] Connection header received
0808 06:09:14.049167 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:14.049169 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:14.049208 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:14.049283 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:14.049836 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:14.050610 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:14.051590 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:14.051883 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:14.051983 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:14.053906 (+ 1923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:14.053910 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:14.053913 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:09:14.053939 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:14.055489 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:14.055891 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:14.055895 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:14.055896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:14.055987 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:14.056267 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:14.056270 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:14.056271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:14.056441 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:14.056535 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:14.056811 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 06:09:14.056944 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 06:09:14.057783 4926 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44323 } has no permanent_uuid. Determining permanent_uuid...
I20260808 06:09:14.057858 4684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:14.049342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38862 (local address 127.1.151.190:44347)
0808 06:09:14.049443 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:14.049446 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:14.049458 (+ 12us) server_negotiation.cc:407] Connection header received
0808 06:09:14.049523 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:14.049526 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:14.049567 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:14.049653 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:14.050141 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:14.050877 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:14.051866 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:14.052151 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:14.052228 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:14.054783 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:14.054786 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:14.054789 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:09:14.054814 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:14.056092 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:14.056488 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:14.056492 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:14.056494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:14.056580 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:14.056843 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:14.056846 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:14.056848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:14.057045 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:14.057282 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:14.057613 (+ 331us) server_negotiation.cc:299] Negotiation successful
0808 06:09:14.057749 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 06:09:14.058471 4801 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44323 } attempt: 3
I20260808 06:09:14.060676 4681 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44323 } attempt: 3
Aug 08 06:09:14 dist-test-slave-zj8q krb5kdc[4557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169353, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.188@KRBTEST.COM for kudu/127.1.151.188@KRBTEST.COM
Aug 08 06:09:14 dist-test-slave-zj8q krb5kdc[4557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.189@KRBTEST.COM
Aug 08 06:09:14 dist-test-slave-zj8q krb5kdc[4557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169353, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.189@KRBTEST.COM for kudu/127.1.151.188@KRBTEST.COM
I20260808 06:09:14.066926 4928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:14.058287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:44323 (local address 127.0.0.1:41874)
0808 06:09:14.058442 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:14.058454 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 06:09:14.058539 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:09:14.058907 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:09:14.058909 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:09:14.059066 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:09:14.059160 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:09:14.059167 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:09:14.059943 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:09:14.059946 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:09:14.060742 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:09:14.060753 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:09:14.061443 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:09:14.061446 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:09:14.061517 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:14.061900 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:09:14.061913 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:09:14.063148 (+ 1235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:09:14.065226 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:09:14.065228 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:09:14.065230 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:09:14.065444 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:09:14.065730 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:09:14.065732 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:09:14.065734 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:09:14.065828 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:09:14.066475 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:09:14.066480 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:09:14.066574 (+ 94us) client_negotiation.cc:769] Sending connection context
0808 06:09:14.066679 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 06:09:14.066795 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":22}
I20260808 06:09:14.067445 4930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:14.058323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41874 (local address 127.1.151.188:44323)
0808 06:09:14.058418 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:14.058422 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:14.058523 (+ 101us) server_negotiation.cc:407] Connection header received
0808 06:09:14.058715 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:14.058718 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:14.058757 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:14.058813 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:14.059264 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:14.059844 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:14.060913 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:14.061270 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:14.061348 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:14.063280 (+ 1932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:14.063284 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:14.063286 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:09:14.063305 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:14.065112 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:14.065557 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:14.065560 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:14.065562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:14.065651 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:14.065919 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:14.065922 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:14.065924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:14.066097 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:14.066221 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:14.067169 (+ 948us) server_negotiation.cc:299] Negotiation successful
0808 06:09:14.067344 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
Aug 08 06:09:14 dist-test-slave-zj8q krb5kdc[4557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169353, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for kudu/127.1.151.188@KRBTEST.COM
I20260808 06:09:14.069198 4807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:14.058842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:44323 (local address 127.0.0.1:41878)
0808 06:09:14.059010 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:14.059024 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:09:14.059095 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:09:14.059542 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:09:14.059545 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:09:14.059719 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:09:14.059802 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:09:14.059809 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:09:14.061227 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:09:14.061230 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:09:14.062252 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:09:14.062264 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:09:14.062965 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:09:14.062969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:09:14.063055 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:14.064073 (+ 1018us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:09:14.064088 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:09:14.065561 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:09:14.067629 (+ 2068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:09:14.067632 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:09:14.067634 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:09:14.067857 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:09:14.068164 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:09:14.068167 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:09:14.068169 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:09:14.068256 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:09:14.068716 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:09:14.068719 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:09:14.068818 (+ 99us) client_negotiation.cc:769] Sending connection context
0808 06:09:14.068907 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 06:09:14.069031 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 06:09:14.069239 4932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:14.058991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41878 (local address 127.1.151.188:44323)
0808 06:09:14.059343 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:14.059351 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:14.059361 (+ 10us) server_negotiation.cc:407] Connection header received
0808 06:09:14.059388 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:14.059391 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:14.059426 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:14.059516 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:14.059910 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:14.061111 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:14.062446 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:14.062795 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:14.062901 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:14.065693 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:14.065697 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:14.065700 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:09:14.065722 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:14.067503 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:14.067964 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:14.067967 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:14.067968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:14.068049 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:14.068354 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:14.068357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:14.068359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:14.068594 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:14.068674 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:14.068975 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 06:09:14.069071 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":281,"thread_start_us":117,"threads_started":1}
I20260808 06:09:14.070261 4812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:14.059326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42500 (local address 127.1.151.189:43243)
0808 06:09:14.059423 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:14.059427 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:14.059437 (+ 10us) server_negotiation.cc:407] Connection header received
0808 06:09:14.059466 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:14.059469 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:14.059504 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:14.059566 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:14.060245 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:14.060737 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:14.061654 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:14.061875 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:14.061956 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:14.064460 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:14.064464 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:14.064467 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:09:14.064490 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:14.066337 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:14.068248 (+ 1911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:14.068251 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:14.068252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:14.068318 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:14.068616 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:14.068620 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:14.068628 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:14.069634 (+ 1006us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:14.069821 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:14.069999 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 06:09:14.070104 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":783,"server-negotiator.queue_time_us":39}
I20260808 06:09:14.070920 4926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ba0e7f5b9784b6ea29877a31ddbbe26: Bootstrap starting.
I20260808 06:09:14.071815 4685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:14.061006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:44323 (local address 127.0.0.1:41880)
0808 06:09:14.061236 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:14.061248 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 06:09:14.061314 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:09:14.063739 (+ 2425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:09:14.063742 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:09:14.063905 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:09:14.063993 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:09:14.064000 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:09:14.064888 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:09:14.064891 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:09:14.065682 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:09:14.065695 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:09:14.066348 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:09:14.066352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:09:14.066438 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:14.066812 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:09:14.066826 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:09:14.068145 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:09:14.069979 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:09:14.069982 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:09:14.069984 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:09:14.070267 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:09:14.070572 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:09:14.070575 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:09:14.070577 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:09:14.070668 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:09:14.071379 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:09:14.071382 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:09:14.071472 (+ 90us) client_negotiation.cc:769] Sending connection context
0808 06:09:14.071552 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 06:09:14.071674 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161}
I20260808 06:09:14.072332 4933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:14.061425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41880 (local address 127.1.151.188:44323)
0808 06:09:14.063091 (+ 1666us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:14.063111 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:14.063123 (+ 12us) server_negotiation.cc:407] Connection header received
0808 06:09:14.063155 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:14.063157 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:14.063194 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:14.063713 (+ 519us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:14.064116 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:14.064761 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:14.065832 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:14.066218 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:14.066624 (+ 406us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:14.068255 (+ 1631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:14.068259 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:14.068262 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:09:14.068284 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:14.069848 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:14.070388 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:14.070390 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:14.070392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:14.070472 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:14.070781 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:14.070783 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:14.070785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:14.071263 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:14.071982 (+ 719us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:14.072055 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 06:09:14.072179 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1605,"thread_start_us":90,"threads_started":1}
I20260808 06:09:14.072587 4801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37b4fae21b784ecf850f14670cf81e44: Bootstrap starting.
I20260808 06:09:14.073572 4926 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ba0e7f5b9784b6ea29877a31ddbbe26: Neither blocks nor log segments found. Creating new log.
I20260808 06:09:14.074283 4926 log.cc:824] T 00000000000000000000000000000000 P 7ba0e7f5b9784b6ea29877a31ddbbe26: Log is configured to *not* fsync() on all Append() calls
I20260808 06:09:14.074545 4801 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 37b4fae21b784ecf850f14670cf81e44: Neither blocks nor log segments found. Creating new log.
I20260808 06:09:14.075323 4801 log.cc:824] T 00000000000000000000000000000000 P 37b4fae21b784ecf850f14670cf81e44: Log is configured to *not* fsync() on all Append() calls
I20260808 06:09:14.076334 4926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ba0e7f5b9784b6ea29877a31ddbbe26: No bootstrap required, opened a new log
I20260808 06:09:14.077826 1630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 06:09:14.078279 4681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c47b9213bb0b4faf97471d591b1dbf49: Bootstrap starting.
I20260808 06:09:14.079201 4644 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "c47b9213bb0b4faf97471d591b1dbf49"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38862
I20260808 06:09:14.080675 4681 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c47b9213bb0b4faf97471d591b1dbf49: Neither blocks nor log segments found. Creating new log.
I20260808 06:09:14.080876 4926 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ba0e7f5b9784b6ea29877a31ddbbe26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c47b9213bb0b4faf97471d591b1dbf49" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44347 } } peers { permanent_uuid: "37b4fae21b784ecf850f14670cf81e44" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 43243 } } peers { permanent_uuid: "7ba0e7f5b9784b6ea29877a31ddbbe26" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44323 } }
I20260808 06:09:14.081115 4926 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ba0e7f5b9784b6ea29877a31ddbbe26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ba0e7f5b9784b6ea29877a31ddbbe26, State: Initialized, Role: FOLLOWER
I20260808 06:09:14.081460 4681 log.cc:824] T 00000000000000000000000000000000 P c47b9213bb0b4faf97471d591b1dbf49: Log is configured to *not* fsync() on all Append() calls
I20260808 06:09:14.081434 4926 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ba0e7f5b9784b6ea29877a31ddbbe26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c47b9213bb0b4faf97471d591b1dbf49" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44347 } } peers { permanent_uuid: "37b4fae21b784ecf850f14670cf81e44" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 43243 } } peers { permanent_uuid: "7ba0e7f5b9784b6ea29877a31ddbbe26" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44323 } }
I20260808 06:09:14.081893 4938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ba0e7f5b9784b6ea29877a31ddbbe26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c47b9213bb0b4faf97471d591b1dbf49" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44347 } } peers { permanent_uuid: "37b4fae21b784ecf850f14670cf81e44" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 43243 } } peers { permanent_uuid: "7ba0e7f5b9784b6ea29877a31ddbbe26" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44323 } } }
I20260808 06:09:14.082048 4938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ba0e7f5b9784b6ea29877a31ddbbe26 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 06:09:14.082795 4926 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ba0e7f5b9784b6ea29877a31ddbbe26 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:09:14.083189 4801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37b4fae21b784ecf850f14670cf81e44: No bootstrap required, opened a new log
I20260808 06:09:14.083616 4681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c47b9213bb0b4faf97471d591b1dbf49: No bootstrap required, opened a new log
I20260808 06:09:14.086426 4681 raft_consensus.cc:348] T 00000000000000000000000000000000 P c47b9213bb0b4faf97471d591b1dbf49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c47b9213bb0b4faf97471d591b1dbf49" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44347 } } peers { permanent_uuid: "37b4fae21b784ecf850f14670cf81e44" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 43243 } } peers { permanent_uuid: "7ba0e7f5b9784b6ea29877a31ddbbe26" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44323 } }
I20260808 06:09:14.086656 4681 raft_consensus.cc:729] T 00000000000000000000000000000000 P c47b9213bb0b4faf97471d591b1dbf49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c47b9213bb0b4faf97471d591b1dbf49, State: Initialized, Role: FOLLOWER
I20260808 06:09:14.086962 4681 consensus_queue.cc:260] T 00000000000000000000000000000000 P c47b9213bb0b4faf97471d591b1dbf49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c47b9213bb0b4faf97471d591b1dbf49" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44347 } } peers { permanent_uuid: "37b4fae21b784ecf850f14670cf81e44" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 43243 } } peers { permanent_uuid: "7ba0e7f5b9784b6ea29877a31ddbbe26" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44323 } }
I20260808 06:09:14.087492 4962 sys_catalog.cc:455] T 00000000000000000000000000000000 P c47b9213bb0b4faf97471d591b1dbf49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c47b9213bb0b4faf97471d591b1dbf49" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44347 } } peers { permanent_uuid: "37b4fae21b784ecf850f14670cf81e44" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 43243 } } peers { permanent_uuid: "7ba0e7f5b9784b6ea29877a31ddbbe26" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44323 } } }
I20260808 06:09:14.087699 4962 sys_catalog.cc:458] T 00000000000000000000000000000000 P c47b9213bb0b4faf97471d591b1dbf49 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 06:09:14.088080 4957 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7ba0e7f5b9784b6ea29877a31ddbbe26: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 06:09:14.088212 4957 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 06:09:14.088222 4681 sys_catalog.cc:564] T 00000000000000000000000000000000 P c47b9213bb0b4faf97471d591b1dbf49 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:09:14.088284 4644 raft_consensus.cc:482] T 00000000000000000000000000000000 P c47b9213bb0b4faf97471d591b1dbf49 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 06:09:14.088392 4644 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c47b9213bb0b4faf97471d591b1dbf49 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:09:14.086400 4813 master_runner.cc:186] Error getting master registration for 127.1.151.190:44347: OK, instance_id { permanent_uuid: "c47b9213bb0b4faf97471d591b1dbf49" instance_seqno: 1786169353600498 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 06:09:14.089548 4813 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 06:09:14.089567 4644 raft_consensus.cc:504] T 00000000000000000000000000000000 P c47b9213bb0b4faf97471d591b1dbf49 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c47b9213bb0b4faf97471d591b1dbf49" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44347 } } peers { permanent_uuid: "37b4fae21b784ecf850f14670cf81e44" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 43243 } } peers { permanent_uuid: "7ba0e7f5b9784b6ea29877a31ddbbe26" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44323 } }
I20260808 06:09:14.090517 4644 leader_election.cc:288] T 00000000000000000000000000000000 P c47b9213bb0b4faf97471d591b1dbf49 [CANDIDATE]: Term 1 election: Requested vote from peers 37b4fae21b784ecf850f14670cf81e44 (127.1.151.189:43243), 7ba0e7f5b9784b6ea29877a31ddbbe26 (127.1.151.188:44323)
I20260808 06:09:14.091048 4881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c47b9213bb0b4faf97471d591b1dbf49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7ba0e7f5b9784b6ea29877a31ddbbe26"
I20260808 06:09:14.091252 4881 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ba0e7f5b9784b6ea29877a31ddbbe26 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:09:14.092494 4881 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7ba0e7f5b9784b6ea29877a31ddbbe26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c47b9213bb0b4faf97471d591b1dbf49 in term 1.
I20260808 06:09:14.093055 4599 leader_election.cc:302] T 00000000000000000000000000000000 P c47b9213bb0b4faf97471d591b1dbf49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ba0e7f5b9784b6ea29877a31ddbbe26, c47b9213bb0b4faf97471d591b1dbf49; no voters:
W20260808 06:09:14.093740 4978 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c47b9213bb0b4faf97471d591b1dbf49: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260808 06:09:14.093811 4962 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c47b9213bb0b4faf97471d591b1dbf49 [term 1 FOLLOWER]: Leader election won for term 1
W20260808 06:09:14.093823 4978 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 06:09:14.094012 4962 raft_consensus.cc:686] T 00000000000000000000000000000000 P c47b9213bb0b4faf97471d591b1dbf49 [term 1 LEADER]: Becoming Leader. State: Replica: c47b9213bb0b4faf97471d591b1dbf49, State: Running, Role: LEADER
I20260808 06:09:14.094349 4962 consensus_queue.cc:237] T 00000000000000000000000000000000 P c47b9213bb0b4faf97471d591b1dbf49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c47b9213bb0b4faf97471d591b1dbf49" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44347 } } peers { permanent_uuid: "37b4fae21b784ecf850f14670cf81e44" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 43243 } } peers { permanent_uuid: "7ba0e7f5b9784b6ea29877a31ddbbe26" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44323 } }
I20260808 06:09:14.096069 4962 sys_catalog.cc:455] T 00000000000000000000000000000000 P c47b9213bb0b4faf97471d591b1dbf49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c47b9213bb0b4faf97471d591b1dbf49. Latest consensus state: current_term: 1 leader_uuid: "c47b9213bb0b4faf97471d591b1dbf49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c47b9213bb0b4faf97471d591b1dbf49" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44347 } } peers { permanent_uuid: "37b4fae21b784ecf850f14670cf81e44" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 43243 } } peers { permanent_uuid: "7ba0e7f5b9784b6ea29877a31ddbbe26" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44323 } } }
I20260808 06:09:14.096186 4962 sys_catalog.cc:458] T 00000000000000000000000000000000 P c47b9213bb0b4faf97471d591b1dbf49 [sys.catalog]: This master's current role is: LEADER
I20260808 06:09:14.096904 4801 raft_consensus.cc:348] T 00000000000000000000000000000000 P 37b4fae21b784ecf850f14670cf81e44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c47b9213bb0b4faf97471d591b1dbf49" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44347 } } peers { permanent_uuid: "37b4fae21b784ecf850f14670cf81e44" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 43243 } } peers { permanent_uuid: "7ba0e7f5b9784b6ea29877a31ddbbe26" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44323 } }
I20260808 06:09:14.097143 4801 raft_consensus.cc:729] T 00000000000000000000000000000000 P 37b4fae21b784ecf850f14670cf81e44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37b4fae21b784ecf850f14670cf81e44, State: Initialized, Role: FOLLOWER
I20260808 06:09:14.097440 4801 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37b4fae21b784ecf850f14670cf81e44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c47b9213bb0b4faf97471d591b1dbf49" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44347 } } peers { permanent_uuid: "37b4fae21b784ecf850f14670cf81e44" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 43243 } } peers { permanent_uuid: "7ba0e7f5b9784b6ea29877a31ddbbe26" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44323 } }
I20260808 06:09:14.094830 4573 master_runner.cc:186] Error getting master registration for 127.1.151.189:43243: OK, instance_id { permanent_uuid: "37b4fae21b784ecf850f14670cf81e44" instance_seqno: 1786169353806640 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 06:09:14.097921 4573 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 06:09:14.098069 4607 tablet.cc:2369] T 00000000000000000000000000000000 P c47b9213bb0b4faf97471d591b1dbf49: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 06:09:14.098783 4801 sys_catalog.cc:564] T 00000000000000000000000000000000 P 37b4fae21b784ecf850f14670cf81e44 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:09:14.098817 4759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c47b9213bb0b4faf97471d591b1dbf49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "37b4fae21b784ecf850f14670cf81e44"
I20260808 06:09:14.099010 4759 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 37b4fae21b784ecf850f14670cf81e44 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:09:14.099332 4991 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 06:09:14.100303 4759 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 37b4fae21b784ecf850f14670cf81e44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c47b9213bb0b4faf97471d591b1dbf49 in term 1.
I20260808 06:09:14.101306 4991 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:09:14.102880 4998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37b4fae21b784ecf850f14670cf81e44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c47b9213bb0b4faf97471d591b1dbf49" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44347 } } peers { permanent_uuid: "37b4fae21b784ecf850f14670cf81e44" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 43243 } } peers { permanent_uuid: "7ba0e7f5b9784b6ea29877a31ddbbe26" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44323 } } }
I20260808 06:09:14.103008 4998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37b4fae21b784ecf850f14670cf81e44 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 06:09:14.109210 5015 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 37b4fae21b784ecf850f14670cf81e44: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 06:09:14.109303 5015 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 06:09:14.121265 4684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:14.093455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38872 (local address 127.1.151.190:44347)
0808 06:09:14.093758 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:14.093763 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:14.103574 (+ 9811us) server_negotiation.cc:407] Connection header received
0808 06:09:14.103767 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:14.103770 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:14.104073 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:14.104171 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:14.105829 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:14.106840 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:14.108197 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:14.108504 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:14.108579 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:14.113698 (+ 5119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:14.113703 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:14.113706 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:09:14.113734 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:14.115438 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:14.115937 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:14.115942 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:14.115944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:14.116033 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:14.116361 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:14.116364 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:14.116366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:14.116590 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:14.116692 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:14.120961 (+ 4269us) server_negotiation.cc:299] Negotiation successful
0808 06:09:14.121088 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232}
I20260808 06:09:14.122716 4881 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7ba0e7f5b9784b6ea29877a31ddbbe26 [term 1 FOLLOWER]: Refusing update from remote peer c47b9213bb0b4faf97471d591b1dbf49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:09:14.123473 4988 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c47b9213bb0b4faf97471d591b1dbf49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ba0e7f5b9784b6ea29877a31ddbbe26" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:09:14.123735 4812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:14.101750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42516 (local address 127.1.151.189:43243)
0808 06:09:14.101852 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:14.101856 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:14.108666 (+ 6810us) server_negotiation.cc:407] Connection header received
0808 06:09:14.108852 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:14.108855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:14.108901 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:14.108949 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:14.114259 (+ 5310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:14.115018 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:14.116163 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:14.116502 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:14.116572 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:14.118367 (+ 1795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:14.118371 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:14.118374 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:09:14.118400 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:14.119993 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:14.120501 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:14.120505 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:14.120507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:14.120601 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:14.121712 (+ 1111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:14.121716 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:14.121717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:14.121932 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:14.122037 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:14.123417 (+ 1380us) server_negotiation.cc:299] Negotiation successful
0808 06:09:14.123582 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 06:09:14.127614 4938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ba0e7f5b9784b6ea29877a31ddbbe26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c47b9213bb0b4faf97471d591b1dbf49. Latest consensus state: current_term: 1 leader_uuid: "c47b9213bb0b4faf97471d591b1dbf49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c47b9213bb0b4faf97471d591b1dbf49" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44347 } } peers { permanent_uuid: "37b4fae21b784ecf850f14670cf81e44" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 43243 } } peers { permanent_uuid: "7ba0e7f5b9784b6ea29877a31ddbbe26" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44323 } } }
I20260808 06:09:14.127733 4938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ba0e7f5b9784b6ea29877a31ddbbe26 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 06:09:14.128695 4759 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 37b4fae21b784ecf850f14670cf81e44 [term 1 FOLLOWER]: Refusing update from remote peer c47b9213bb0b4faf97471d591b1dbf49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:09:14.129119 4933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:14.102439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41890 (local address 127.1.151.188:44323)
0808 06:09:14.102558 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:14.102563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:14.109465 (+ 6902us) server_negotiation.cc:407] Connection header received
0808 06:09:14.110821 (+ 1356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:14.110825 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:14.110876 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:14.110926 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:14.111716 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:14.112492 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:14.122127 (+ 9635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:14.122494 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:14.122566 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:14.125648 (+ 3082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:14.125651 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:14.125654 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:09:14.125682 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:14.127335 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:14.127794 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:14.127797 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:14.127799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:14.127885 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:14.128187 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:14.128190 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:14.128192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:14.128407 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:14.128501 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:14.128846 (+ 345us) server_negotiation.cc:299] Negotiation successful
0808 06:09:14.128963 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 06:09:14.129540 4988 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c47b9213bb0b4faf97471d591b1dbf49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37b4fae21b784ecf850f14670cf81e44" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 43243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:09:14.133435 4998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37b4fae21b784ecf850f14670cf81e44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c47b9213bb0b4faf97471d591b1dbf49. Latest consensus state: current_term: 1 leader_uuid: "c47b9213bb0b4faf97471d591b1dbf49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c47b9213bb0b4faf97471d591b1dbf49" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44347 } } peers { permanent_uuid: "37b4fae21b784ecf850f14670cf81e44" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 43243 } } peers { permanent_uuid: "7ba0e7f5b9784b6ea29877a31ddbbe26" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44323 } } }
I20260808 06:09:14.133571 4998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37b4fae21b784ecf850f14670cf81e44 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 06:09:14.143582 4962 sys_catalog.cc:455] T 00000000000000000000000000000000 P c47b9213bb0b4faf97471d591b1dbf49 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c47b9213bb0b4faf97471d591b1dbf49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c47b9213bb0b4faf97471d591b1dbf49" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44347 } } peers { permanent_uuid: "37b4fae21b784ecf850f14670cf81e44" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 43243 } } peers { permanent_uuid: "7ba0e7f5b9784b6ea29877a31ddbbe26" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44323 } } }
I20260808 06:09:14.143702 4962 sys_catalog.cc:458] T 00000000000000000000000000000000 P c47b9213bb0b4faf97471d591b1dbf49 [sys.catalog]: This master's current role is: LEADER
I20260808 06:09:14.143788 4962 sys_catalog.cc:455] T 00000000000000000000000000000000 P c47b9213bb0b4faf97471d591b1dbf49 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c47b9213bb0b4faf97471d591b1dbf49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c47b9213bb0b4faf97471d591b1dbf49" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44347 } } peers { permanent_uuid: "37b4fae21b784ecf850f14670cf81e44" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 43243 } } peers { permanent_uuid: "7ba0e7f5b9784b6ea29877a31ddbbe26" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44323 } } }
I20260808 06:09:14.143848 4962 sys_catalog.cc:458] T 00000000000000000000000000000000 P c47b9213bb0b4faf97471d591b1dbf49 [sys.catalog]: This master's current role is: LEADER
I20260808 06:09:14.143980 4938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ba0e7f5b9784b6ea29877a31ddbbe26 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c47b9213bb0b4faf97471d591b1dbf49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c47b9213bb0b4faf97471d591b1dbf49" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44347 } } peers { permanent_uuid: "37b4fae21b784ecf850f14670cf81e44" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 43243 } } peers { permanent_uuid: "7ba0e7f5b9784b6ea29877a31ddbbe26" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44323 } } }
I20260808 06:09:14.144085 4938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ba0e7f5b9784b6ea29877a31ddbbe26 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 06:09:14.145469 4998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37b4fae21b784ecf850f14670cf81e44 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c47b9213bb0b4faf97471d591b1dbf49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c47b9213bb0b4faf97471d591b1dbf49" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 44347 } } peers { permanent_uuid: "37b4fae21b784ecf850f14670cf81e44" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 43243 } } peers { permanent_uuid: "7ba0e7f5b9784b6ea29877a31ddbbe26" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44323 } } }
I20260808 06:09:14.145599 4998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37b4fae21b784ecf850f14670cf81e44 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 06:09:14.145706 4991 catalog_manager.cc:1347] Generated new cluster ID: 82820323fbd1446c83653e6f6ee7daa1
I20260808 06:09:14.145767 4991 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:09:14.155414 5021 mvcc.cc:205] Tried to move back new op lower bound from 7316149674469572608 to 7316149674373976064. Current Snapshot: MvccSnapshot[applied={T|T < 7316149674469572608}]
I20260808 06:09:14.175174 4991 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:09:14.176483 4991 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:09:14.196993 4991 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c47b9213bb0b4faf97471d591b1dbf49: Generated new TSK 0
I20260808 06:09:14.198031 4991 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 06:09:14.235090 4684 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:09:14.231815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38874 (local address 127.1.151.190:44347)
0808 06:09:14.231960 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:14.231965 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:14.233922 (+ 1957us) server_negotiation.cc:407] Connection header received
0808 06:09:14.234145 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:14.234184 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:14.234242 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:14.234298 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:14.234954 (+ 656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38874: BlockingRecv error: recv got EOF from 127.0.0.1:38874 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 06:09:14.236660 4812 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:09:14.232655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42530 (local address 127.1.151.189:43243)
0808 06:09:14.232776 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:14.232780 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:14.235761 (+ 2981us) server_negotiation.cc:407] Connection header received
0808 06:09:14.235935 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:14.235938 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:14.235988 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:14.236034 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:14.236542 (+ 508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42530: BlockingRecv error: recv got EOF from 127.0.0.1:42530 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260808 06:09:14.238289 4933 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:09:14.233325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41902 (local address 127.1.151.188:44323)
0808 06:09:14.233440 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:14.233444 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:14.237297 (+ 3853us) server_negotiation.cc:407] Connection header received
0808 06:09:14.237482 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:14.237486 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:14.237537 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:14.237587 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:14.238087 (+ 500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41902: BlockingRecv error: recv got EOF from 127.0.0.1:41902 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 06:09:14.242923 4812 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:09:14.241379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42546 (local address 127.1.151.189:43243)
0808 06:09:14.241478 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:14.241482 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:14.241912 (+ 430us) server_negotiation.cc:407] Connection header received
0808 06:09:14.242132 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:14.242166 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:14.242257 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:14.242315 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:14.242827 (+ 512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42546: BlockingRecv error: recv got EOF from 127.0.0.1:42546 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260808 06:09:14.248350 4812 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:09:14.246853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42560 (local address 127.1.151.189:43243)
0808 06:09:14.246955 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:14.246959 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:14.247446 (+ 487us) server_negotiation.cc:407] Connection header received
0808 06:09:14.247642 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:14.247646 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:14.247696 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:14.247748 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:14.248262 (+ 514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42560: BlockingRecv error: recv got EOF from 127.0.0.1:42560 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 06:09:14.254077 4812 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:09:14.252651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42564 (local address 127.1.151.189:43243)
0808 06:09:14.252753 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:14.252757 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:14.253229 (+ 472us) server_negotiation.cc:407] Connection header received
0808 06:09:14.253408 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:14.253410 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:14.253456 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:14.253506 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:14.253986 (+ 480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42564: BlockingRecv error: recv got EOF from 127.0.0.1:42564 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 06:09:14.261917 4812 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:09:14.260393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42580 (local address 127.1.151.189:43243)
0808 06:09:14.260524 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:14.260528 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:14.261009 (+ 481us) server_negotiation.cc:407] Connection header received
0808 06:09:14.261221 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:14.261225 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:14.261276 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:14.261327 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:14.261827 (+ 500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42580: BlockingRecv error: recv got EOF from 127.0.0.1:42580 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260808 06:09:14.273909 4812 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:09:14.272421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42584 (local address 127.1.151.189:43243)
0808 06:09:14.272528 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:14.272532 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:14.272982 (+ 450us) server_negotiation.cc:407] Connection header received
0808 06:09:14.273162 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:14.273165 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:14.273212 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:14.273261 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:14.273808 (+ 547us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42584: BlockingRecv error: recv got EOF from 127.0.0.1:42584 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260808 06:09:14.293495 4812 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:09:14.292288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42592 (local address 127.1.151.189:43243)
0808 06:09:14.292402 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:14.292406 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:14.292640 (+ 234us) server_negotiation.cc:407] Connection header received
0808 06:09:14.292771 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:14.292774 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:14.292824 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:14.292890 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:14.293381 (+ 491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42592: BlockingRecv error: recv got EOF from 127.0.0.1:42592 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 06:09:14.328867 4812 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:09:14.327729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42602 (local address 127.1.151.189:43243)
0808 06:09:14.327867 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:14.327872 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:14.327940 (+ 68us) server_negotiation.cc:407] Connection header received
0808 06:09:14.328107 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:14.328111 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:14.328160 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:14.328229 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:14.328759 (+ 530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42602: BlockingRecv error: recv got EOF from 127.0.0.1:42602 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260808 06:09:14.396121 4812 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:09:14.395043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42608 (local address 127.1.151.189:43243)
0808 06:09:14.395195 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:14.395201 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:14.395357 (+ 156us) server_negotiation.cc:407] Connection header received
0808 06:09:14.395476 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:14.395479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:14.395528 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:14.395601 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:14.395980 (+ 379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42608: BlockingRecv error: recv got EOF from 127.0.0.1:42608 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260808 06:09:14.527406 4812 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:09:14.526258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42616 (local address 127.1.151.189:43243)
0808 06:09:14.526410 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:14.526415 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:14.526533 (+ 118us) server_negotiation.cc:407] Connection header received
0808 06:09:14.526741 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:14.526743 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:14.526795 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:14.526857 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:14.527275 (+ 418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42616: BlockingRecv error: recv got EOF from 127.0.0.1:42616 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 06:09:14.786800 4812 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:09:14.785692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42620 (local address 127.1.151.189:43243)
0808 06:09:14.785827 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:14.785831 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:14.785881 (+ 50us) server_negotiation.cc:407] Connection header received
0808 06:09:14.786008 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:14.786010 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:14.786077 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:14.786183 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:14.786674 (+ 491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42620: BlockingRecv error: recv got EOF from 127.0.0.1:42620 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 06:09:15.089851 4957 catalog_manager.cc:1259] Loaded cluster ID: 82820323fbd1446c83653e6f6ee7daa1
I20260808 06:09:15.089947 4957 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7ba0e7f5b9784b6ea29877a31ddbbe26: loading cluster ID for follower catalog manager: success
I20260808 06:09:15.091687 4957 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7ba0e7f5b9784b6ea29877a31ddbbe26: acquiring CA information for follower catalog manager: success
I20260808 06:09:15.092846 4957 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7ba0e7f5b9784b6ea29877a31ddbbe26: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 06:09:15.110807 5015 catalog_manager.cc:1259] Loaded cluster ID: 82820323fbd1446c83653e6f6ee7daa1
I20260808 06:09:15.110875 5015 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 37b4fae21b784ecf850f14670cf81e44: loading cluster ID for follower catalog manager: success
I20260808 06:09:15.112510 5015 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 37b4fae21b784ecf850f14670cf81e44: acquiring CA information for follower catalog manager: success
I20260808 06:09:15.113674 5015 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 37b4fae21b784ecf850f14670cf81e44: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 06:09:15.304797 5135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:15.300879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42632 (local address 127.1.151.189:43243)
0808 06:09:15.301123 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:15.301129 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:15.301214 (+ 85us) server_negotiation.cc:407] Connection header received
0808 06:09:15.301383 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:15.301387 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:15.301522 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:15.301621 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:09:15.301922 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:15.302479 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:15.303330 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:15.303559 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:15.303649 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:15.303962 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:09:15.304400 (+ 438us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:09:15.304509 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:15.304582 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 06:09:15.304642 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":68,"threads_started":1}
I20260808 06:09:15.311192 5152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:15.307827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41914 (local address 127.1.151.188:44323)
0808 06:09:15.308049 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:15.308053 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:15.308068 (+ 15us) server_negotiation.cc:407] Connection header received
0808 06:09:15.308148 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:15.308150 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:15.308260 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:15.308351 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:09:15.308501 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:15.309036 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:15.309823 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:15.310056 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:15.310145 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:15.310411 (+ 266us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:09:15.310849 (+ 438us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:09:15.310951 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:15.310978 (+ 27us) server_negotiation.cc:299] Negotiation successful
0808 06:09:15.311029 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":78,"threads_started":1}
I20260808 06:09:15.317834 5135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:15.313787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42644 (local address 127.1.151.189:43243)
0808 06:09:15.313891 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:15.313896 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:15.314326 (+ 430us) server_negotiation.cc:407] Connection header received
0808 06:09:15.314508 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:15.314512 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:15.314567 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:15.314621 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:09:15.314958 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:15.315475 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:15.316723 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:15.317040 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:15.317138 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:15.317391 (+ 253us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:09:15.317455 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:09:15.317516 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:15.317655 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 06:09:15.317708 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 06:09:15.317866 5152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:15.314309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41918 (local address 127.1.151.188:44323)
0808 06:09:15.314415 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:15.314418 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:15.314613 (+ 195us) server_negotiation.cc:407] Connection header received
0808 06:09:15.314749 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:15.314752 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:15.314790 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:15.314846 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:09:15.315091 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:15.315790 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:15.316642 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:15.316949 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:15.317047 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:15.317356 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:09:15.317430 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:09:15.317506 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:15.317672 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 06:09:15.317740 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 06:09:15.319085 5160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:15.313805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38888 (local address 127.1.151.190:44347)
0808 06:09:15.314033 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:15.314036 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:15.314063 (+ 27us) server_negotiation.cc:407] Connection header received
0808 06:09:15.314177 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:15.314180 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:15.314334 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:15.314496 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:09:15.314728 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:15.315495 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:15.317133 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:15.317436 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:15.317565 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:15.317866 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:09:15.318323 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:09:15.318407 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:15.318543 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 06:09:15.318602 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":82,"threads_started":1}
I20260808 06:09:15.321868 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 4573
I20260808 06:09:15.335599 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 4694
I20260808 06:09:15.347086 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 4813
2026-08-08T06:09:15Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3200 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 06:09:15 dist-test-slave-zj8q krb5kdc[5168](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 06:09:15 dist-test-slave-zj8q krb5kdc[5168](info): set up 2 sockets
Aug 08 06:09:15 dist-test-slave-zj8q krb5kdc[5168](info): commencing operation
krb5kdc: starting...
W20260808 06:09:16.490661 1630 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/krb5kdc/test-user.keytab.
Aug 08 06:09:16 dist-test-slave-zj8q krb5kdc[5168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T06:09:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T06:09:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.151.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
Entry for principal kudu/127.1.151.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab.
I20260808 06:09:16.611264 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:42471
--webserver_interface=127.1.151.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:44703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:42471,127.1.151.189:37567,127.1.151.188:44523
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 06:09:16.719215 5187 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 06:09:16.719481 5187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:09:16.719522 5187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:09:16.719581 5187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:09:16.723480 5187 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:09:16.723543 5187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:09:16.723567 5187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:09:16.723589 5187 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:09:16.723616 5187 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:09:16.728787 5187 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:44703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:42471,127.1.151.189:37567,127.1.151.188:44523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.190.keytab
--principal=kudu/127.1.151.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.190:42471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.151.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:09:16.729777 5187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:09:16.730787 5187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:09:16.738984 5196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:09:16.739025 5193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:09:16.739030 5194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:09:16.739080 5187 server_base.cc:1034] running on GCE node
I20260808 06:09:16.739880 5187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:09:16.741041 5187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:09:16.742241 5187 hybrid_clock.cc:648] HybridClock initialized: now 1786169356742213 us; error 37 us; skew 500 ppm
Aug 08 06:09:16 dist-test-slave-zj8q krb5kdc[5168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169356, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:09:16.744971 5187 init.cc:377] Logged in from keytab as kudu/127.1.151.190@KRBTEST.COM (short username kudu)
I20260808 06:09:16.745836 5187 webserver.cc:460] Webserver started at http://127.1.151.190:43295/ using document root <none> and password file <none>
I20260808 06:09:16.746441 5187 fs_manager.cc:359] Metadata directory not provided
I20260808 06:09:16.746510 5187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:09:16.746721 5187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:09:16.748801 5187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-0/data/instance:
uuid: "81217163bd8e4a0c9a4408aa2656bcee"
format_stamp: "Formatted at 2026-08-08 06:09:16 on dist-test-slave-zj8q"
server_key: "4d5af9257bf57493d159788e2e32b68b"
server_key_iv: "1bf3a864eda99b01bf926e104bf9f337"
server_key_version: "encryptionkey@0"
I20260808 06:09:16.749353 5187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-0/wal/instance:
uuid: "81217163bd8e4a0c9a4408aa2656bcee"
format_stamp: "Formatted at 2026-08-08 06:09:16 on dist-test-slave-zj8q"
server_key: "4d5af9257bf57493d159788e2e32b68b"
server_key_iv: "1bf3a864eda99b01bf926e104bf9f337"
server_key_version: "encryptionkey@0"
I20260808 06:09:16.752980 5187 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260808 06:09:16.755373 5208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:09:16.756613 5187 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 06:09:16.756736 5187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-0/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-0/wal
uuid: "81217163bd8e4a0c9a4408aa2656bcee"
format_stamp: "Formatted at 2026-08-08 06:09:16 on dist-test-slave-zj8q"
server_key: "4d5af9257bf57493d159788e2e32b68b"
server_key_iv: "1bf3a864eda99b01bf926e104bf9f337"
server_key_version: "encryptionkey@0"
I20260808 06:09:16.756831 5187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:09:16.783800 5187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:09:16.786837 5187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:09:16.787032 5187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:09:16.793838 5187 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.190:42471
I20260808 06:09:16.794117 5285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.190:42471 every 8 connection(s)
I20260808 06:09:16.794839 5187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-0/data/info.pb
I20260808 06:09:16.796998 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 5187
I20260808 06:09:16.797119 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-0/wal/instance
I20260808 06:09:16.797410 1630 external_mini_cluster.cc:1428] Setting key 6770d30f51df5eb9fb7352a404189ca1
I20260808 06:09:16.797633 5288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:09:16.801409 5288 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.151.190" port: 42471 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.151.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.189@KRBTEST.COM" created.
Aug 08 06:09:16 dist-test-slave-zj8q krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169356, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:09:16.816335 5294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:16.803773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33610 (local address 127.1.151.190:42471)
0808 06:09:16.804188 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:16.804206 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:16.804476 (+ 270us) server_negotiation.cc:407] Connection header received
0808 06:09:16.805480 (+ 1004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:16.805484 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:16.805693 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:16.805854 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:16.806695 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:16.807587 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:16.808619 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:16.809023 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:16.809173 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:16.811521 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:16.811528 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:16.811541 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:09:16.811567 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:16.813334 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:16.813793 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:16.813799 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:16.813801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:16.813933 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:16.814333 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:16.814336 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:16.814337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:16.814758 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:16.814872 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:16.815345 (+ 473us) server_negotiation.cc:299] Negotiation successful
0808 06:09:16.815516 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":116,"threads_started":1}
I20260808 06:09:16.816339 5296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:16.803774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:42471 (local address 127.0.0.1:33610)
0808 06:09:16.804331 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:16.804367 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 06:09:16.805184 (+ 817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:09:16.805876 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:09:16.805881 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:09:16.806321 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:09:16.806555 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:09:16.806568 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:09:16.807699 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:09:16.807702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:09:16.808485 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:09:16.808495 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:09:16.809171 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:09:16.809175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:09:16.809290 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:16.809793 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:09:16.809809 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:09:16.811361 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:09:16.813459 (+ 2098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:09:16.813467 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:09:16.813475 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:09:16.813689 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:09:16.814029 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:09:16.814033 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:09:16.814035 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:09:16.814231 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:09:16.814880 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:09:16.814887 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:09:16.815065 (+ 178us) client_negotiation.cc:769] Sending connection context
0808 06:09:16.815244 (+ 179us) client_negotiation.cc:240] Negotiation successful
0808 06:09:16.815638 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"mutex_wait_us":74,"spinlock_wait_cycles":8064,"thread_start_us":159,"threads_started":1}
I20260808 06:09:16.818723 5288 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.151.189" port: 37567 } has no permanent_uuid. Determining permanent_uuid...
I20260808 06:09:16.819473 5296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:16.819188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.189:37567 (local address 127.0.0.1:47040)
0808 06:09:16.819305 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:16.819375 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.189:37567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 06:09:16.819711 5213 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.151.189:37567: connect: Connection refused (error 111)
I20260808 06:09:16.820474 5296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:16.820290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.189:37567 (local address 127.0.0.1:47052)
0808 06:09:16.820383 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:16.820426 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.189:37567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 06:09:16.820652 5288 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.151.189:37567: Network error: Client connection negotiation failed: client connection to 127.1.151.189:37567: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.189.keytab.
Entry for principal kudu/127.1.151.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.189.keytab.
I20260808 06:09:16.825312 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.189:37567
--webserver_interface=127.1.151.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.189.keytab
--principal=kudu/127.1.151.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:44703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:42471,127.1.151.189:37567,127.1.151.188:44523
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 06:09:16.869899 5288 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.151.189" port: 37567 } attempt: 1
I20260808 06:09:16.870826 5296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:16.870511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.189:37567 (local address 127.0.0.1:47062)
0808 06:09:16.870680 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:16.870747 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.189:37567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 06:09:16.871511 5296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:16.871325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.189:37567 (local address 127.0.0.1:47078)
0808 06:09:16.871412 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:16.871455 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.189:37567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 06:09:16.871637 5288 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.151.189:37567: Network error: Client connection negotiation failed: client connection to 127.1.151.189:37567: connect: Connection refused (error 111)
W20260808 06:09:16.936096 5300 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 06:09:16.936347 5300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:09:16.936390 5300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:09:16.936447 5300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 06:09:16.939811 5288 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.151.189" port: 37567 } attempt: 2
W20260808 06:09:16.940448 5300 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:09:16.940513 5300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:09:16.940541 5300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:09:16.940574 5300 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:09:16.940603 5300 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:09:16.940708 5296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:16.940381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.189:37567 (local address 127.0.0.1:47092)
0808 06:09:16.940546 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:16.940624 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.189:37567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 06:09:16.941534 5296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:16.941345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.189:37567 (local address 127.0.0.1:47100)
0808 06:09:16.941432 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:16.941477 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.189:37567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 06:09:16.941650 5288 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.151.189:37567: Network error: Client connection negotiation failed: client connection to 127.1.151.189:37567: connect: Connection refused (error 111)
I20260808 06:09:16.945971 5300 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:44703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:42471,127.1.151.189:37567,127.1.151.188:44523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.189.keytab
--principal=kudu/127.1.151.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.189:37567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.151.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:09:16.946960 5300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:09:16.947919 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 06:09:16.956189 5300 server_base.cc:1034] running on GCE node
W20260808 06:09:16.956148 5308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:09:16.956156 5310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:09:16.956176 5313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:09:16.956883 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:09:16.957907 5300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:09:16.959069 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786169356959041 us; error 39 us; skew 500 ppm
Aug 08 06:09:16 dist-test-slave-zj8q krb5kdc[5168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169356, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:09:16.961711 5300 init.cc:377] Logged in from keytab as kudu/127.1.151.189@KRBTEST.COM (short username kudu)
I20260808 06:09:16.962567 5300 webserver.cc:460] Webserver started at http://127.1.151.189:39797/ using document root <none> and password file <none>
I20260808 06:09:16.963099 5300 fs_manager.cc:359] Metadata directory not provided
I20260808 06:09:16.963188 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:09:16.963406 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:09:16.965551 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-1/data/instance:
uuid: "9c3ac46dd7104dc89dfa7a82e793fec1"
format_stamp: "Formatted at 2026-08-08 06:09:16 on dist-test-slave-zj8q"
server_key: "8c20bc5ffac02855034b8595ece361a5"
server_key_iv: "0f6bd51a748ec21fbb1a940af85838cc"
server_key_version: "encryptionkey@0"
I20260808 06:09:16.966087 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-1/wal/instance:
uuid: "9c3ac46dd7104dc89dfa7a82e793fec1"
format_stamp: "Formatted at 2026-08-08 06:09:16 on dist-test-slave-zj8q"
server_key: "8c20bc5ffac02855034b8595ece361a5"
server_key_iv: "0f6bd51a748ec21fbb1a940af85838cc"
server_key_version: "encryptionkey@0"
I20260808 06:09:16.969571 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 06:09:16.971890 5320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:09:16.973034 5300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260808 06:09:16.973148 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-1/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-1/wal
uuid: "9c3ac46dd7104dc89dfa7a82e793fec1"
format_stamp: "Formatted at 2026-08-08 06:09:16 on dist-test-slave-zj8q"
server_key: "8c20bc5ffac02855034b8595ece361a5"
server_key_iv: "0f6bd51a748ec21fbb1a940af85838cc"
server_key_version: "encryptionkey@0"
I20260808 06:09:16.973239 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:09:16.987160 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:09:16.989755 5300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:09:16.989912 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:09:16.996322 5300 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.189:37567
I20260808 06:09:16.996563 5400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.189:37567 every 8 connection(s)
I20260808 06:09:16.997198 5300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-1/data/info.pb
I20260808 06:09:16.999809 5401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:09:17.001843 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 5300
I20260808 06:09:17.001981 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-1/wal/instance
I20260808 06:09:17.002310 1630 external_mini_cluster.cc:1428] Setting key a60a9675d0ea027f2961afbfc6c94b8f
I20260808 06:09:17.003996 5401 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.151.190" port: 42471 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.151.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.151.188@KRBTEST.COM" created.
Aug 08 06:09:17 dist-test-slave-zj8q krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169356, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.189@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:09:17.018847 5294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:17.005907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33612 (local address 127.1.151.190:42471)
0808 06:09:17.006025 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:17.006035 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:17.006979 (+ 944us) server_negotiation.cc:407] Connection header received
0808 06:09:17.007902 (+ 923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:17.007906 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:17.007971 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:17.008040 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:17.009137 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.009691 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.010819 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.011049 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.011145 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:17.014437 (+ 3292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:17.014440 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:17.014442 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:09:17.014468 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:17.016389 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:17.017066 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:17.017070 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:17.017071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:17.017157 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:17.017629 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:17.017632 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:17.017633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:17.017821 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:17.017898 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:17.018581 (+ 683us) server_negotiation.cc:299] Negotiation successful
0808 06:09:17.018720 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 06:09:17.019917 5406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:17.006369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:42471 (local address 127.0.0.1:33612)
0808 06:09:17.006837 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:17.006877 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 06:09:17.007705 (+ 828us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:09:17.008223 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:09:17.008229 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:09:17.008564 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:09:17.008997 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:09:17.009009 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.009812 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.009815 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:09:17.010705 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:09:17.010714 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.011205 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.011209 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:09:17.011362 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:17.011864 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:09:17.011879 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:09:17.014270 (+ 2391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:09:17.016507 (+ 2237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:09:17.016512 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:09:17.016526 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:09:17.016858 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:09:17.017282 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:09:17.017285 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:09:17.017287 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:09:17.017472 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:09:17.017931 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:09:17.017938 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:09:17.018301 (+ 363us) client_negotiation.cc:769] Sending connection context
0808 06:09:17.018514 (+ 213us) client_negotiation.cc:240] Negotiation successful
0808 06:09:17.018868 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":126,"threads_started":1}
I20260808 06:09:17.021347 5401 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.151.189" port: 37567 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 06:09:17 dist-test-slave-zj8q krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169356, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.189@KRBTEST.COM for kudu/127.1.151.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.151.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.188.keytab.
Entry for principal kudu/127.1.151.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.188.keytab.
I20260808 06:09:17.029148 1630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
/tmp/dist-test-taskIVYy3F/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.188:44523
--webserver_interface=127.1.151.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.188.keytab
--principal=kudu/127.1.151.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.151.148:44703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.151.190:42471,127.1.151.189:37567,127.1.151.188:44523
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 06:09:17.029950 5406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:17.021820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.189:37567 (local address 127.0.0.1:47110)
0808 06:09:17.021996 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:17.022006 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 06:09:17.022068 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:09:17.022562 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:09:17.022565 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:09:17.022725 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:09:17.022807 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:09:17.022813 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.023803 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.023806 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:09:17.024351 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:09:17.024359 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.024967 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.024971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:09:17.025062 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:17.025455 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:09:17.025468 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:09:17.026708 (+ 1240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:09:17.028534 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:09:17.028537 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:09:17.028540 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:09:17.028709 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:09:17.028966 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:09:17.028970 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:09:17.028971 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:09:17.029039 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:09:17.029485 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:09:17.029487 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:09:17.029571 (+ 84us) client_negotiation.cc:769] Sending connection context
0808 06:09:17.029663 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 06:09:17.029789 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121}
I20260808 06:09:17.030045 5408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:17.021935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47110 (local address 127.1.151.189:37567)
0808 06:09:17.022256 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:17.022266 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:17.022283 (+ 17us) server_negotiation.cc:407] Connection header received
0808 06:09:17.022316 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:17.022320 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:17.022458 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:17.022569 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:17.022923 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.023695 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.024446 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.024867 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.024965 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:17.026826 (+ 1861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:17.026835 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:17.026847 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:09:17.026868 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:17.028439 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:17.028803 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:17.028809 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:17.028810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:17.028888 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:17.029116 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:17.029119 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:17.029121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:17.029394 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:17.029511 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:17.029758 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 06:09:17.029881 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":83,"threads_started":1}
I20260808 06:09:17.031751 5401 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44523 } has no permanent_uuid. Determining permanent_uuid...
I20260808 06:09:17.032429 5406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:17.032191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:44523 (local address 127.0.0.1:53188)
0808 06:09:17.032301 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:17.032370 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 06:09:17.032639 5322 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44523: connect: Connection refused (error 111)
I20260808 06:09:17.032786 5288 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.151.189" port: 37567 } attempt: 3
I20260808 06:09:17.033440 5406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:17.033253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:44523 (local address 127.0.0.1:53200)
0808 06:09:17.033348 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:17.033388 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 06:09:17.033553 5401 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.151.188:44523: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44523: connect: Connection refused (error 111)
Aug 08 06:09:17 dist-test-slave-zj8q krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169356, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for kudu/127.1.151.189@KRBTEST.COM
I20260808 06:09:17.041216 5408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:17.033751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47126 (local address 127.1.151.189:37567)
0808 06:09:17.033848 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:17.033851 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:17.033862 (+ 11us) server_negotiation.cc:407] Connection header received
0808 06:09:17.033889 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:17.033891 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:17.033924 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:17.033982 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:17.034502 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.035014 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.036068 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.036302 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.036377 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:17.038479 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:17.038482 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:17.038484 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:09:17.038504 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:17.039746 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:17.040206 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:17.040208 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:17.040210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:17.040274 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:17.040524 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:17.040526 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:17.040527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:17.040678 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:17.040746 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:17.041017 (+ 271us) server_negotiation.cc:299] Negotiation successful
0808 06:09:17.041111 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 06:09:17.041383 5296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:17.033188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.189:37567 (local address 127.0.0.1:47126)
0808 06:09:17.033391 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:17.033406 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 06:09:17.033486 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:09:17.034054 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:09:17.034057 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:09:17.034279 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:09:17.034399 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:09:17.034406 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.035114 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.035118 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:09:17.035944 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:09:17.035956 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.036404 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.036408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:09:17.036518 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:17.036928 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:09:17.036941 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:09:17.038346 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:09:17.039854 (+ 1508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:09:17.039858 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:09:17.039860 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:09:17.040119 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:09:17.040348 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:09:17.040351 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:09:17.040353 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:09:17.040448 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:09:17.040763 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:09:17.040767 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:09:17.040876 (+ 109us) client_negotiation.cc:769] Sending connection context
0808 06:09:17.040987 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 06:09:17.041137 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123}
I20260808 06:09:17.042523 5288 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44523 } has no permanent_uuid. Determining permanent_uuid...
I20260808 06:09:17.043200 5296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:17.042963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:44523 (local address 127.0.0.1:53202)
0808 06:09:17.043079 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:17.043131 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 06:09:17.043906 5296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:17.043676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:44523 (local address 127.0.0.1:53206)
0808 06:09:17.043810 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:17.043856 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260808 06:09:17.044016 5288 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.151.188:44523: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44523: connect: Connection refused (error 111)
I20260808 06:09:17.075160 5288 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44523 } attempt: 1
I20260808 06:09:17.076031 5296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:17.075730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:44523 (local address 127.0.0.1:53214)
0808 06:09:17.075886 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:17.075951 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 06:09:17.076756 5296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:17.076510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:44523 (local address 127.0.0.1:53226)
0808 06:09:17.076614 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:17.076684 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 06:09:17.076982 5288 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.151.188:44523: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44523: connect: Connection refused (error 111)
I20260808 06:09:17.082782 5401 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44523 } attempt: 1
I20260808 06:09:17.083443 5406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:17.083204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:44523 (local address 127.0.0.1:53238)
0808 06:09:17.083324 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:17.083376 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 06:09:17.084146 5406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:17.083961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:44523 (local address 127.0.0.1:53244)
0808 06:09:17.084047 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:17.084088 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 06:09:17.084277 5401 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.151.188:44523: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44523: connect: Connection refused (error 111)
W20260808 06:09:17.146633 5409 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 06:09:17.146891 5409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 06:09:17.146912 5409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 06:09:17.146955 5409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 06:09:17.150811 5409 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 06:09:17.150859 5409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 06:09:17.150877 5409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 06:09:17.150892 5409 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 06:09:17.150905 5409 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 06:09:17.152180 5288 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44523 } attempt: 2
I20260808 06:09:17.152477 5401 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44523 } attempt: 2
I20260808 06:09:17.153071 5296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:17.152744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:44523 (local address 127.0.0.1:53258)
0808 06:09:17.152904 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:17.152970 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 06:09:17.153102 5406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:17.152845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:44523 (local address 127.0.0.1:53262)
0808 06:09:17.152975 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:17.153036 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 06:09:17.153849 5406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:17.153603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:44523 (local address 127.0.0.1:53270)
0808 06:09:17.153725 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:17.153779 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 06:09:17.153856 5296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:17.153615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:44523 (local address 127.0.0.1:53286)
0808 06:09:17.153753 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:17.153795 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260808 06:09:17.154021 5401 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.151.188:44523: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44523: connect: Connection refused (error 111)
W20260808 06:09:17.154044 5288 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.151.188:44523: Network error: Client connection negotiation failed: client connection to 127.1.151.188:44523: connect: Connection refused (error 111)
I20260808 06:09:17.156456 5409 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.151.148:44703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.151.190:42471,127.1.151.189:37567,127.1.151.188:44523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/krb5kdc/kudu_127.1.151.188.keytab
--principal=kudu/127.1.151.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.151.188:44523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.151.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 05:27:34 UTC on ccycloud.quasar-rquzrl.root.comops.site
build id 4574260
I20260808 06:09:17.157399 5409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 06:09:17.158419 5409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 06:09:17.166476 5418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:09:17.166501 5419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 06:09:17.166476 5422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 06:09:17.166891 5409 server_base.cc:1034] running on GCE node
I20260808 06:09:17.167434 5409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 06:09:17.168448 5409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 06:09:17.169612 5409 hybrid_clock.cc:648] HybridClock initialized: now 1786169357169591 us; error 44 us; skew 500 ppm
Aug 08 06:09:17 dist-test-slave-zj8q krb5kdc[5168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169357, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 06:09:17.172516 5409 init.cc:377] Logged in from keytab as kudu/127.1.151.188@KRBTEST.COM (short username kudu)
I20260808 06:09:17.173395 5409 webserver.cc:460] Webserver started at http://127.1.151.188:34671/ using document root <none> and password file <none>
I20260808 06:09:17.173923 5409 fs_manager.cc:359] Metadata directory not provided
I20260808 06:09:17.173988 5409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 06:09:17.174212 5409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 06:09:17.176292 5409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-2/data/instance:
uuid: "1f50c2e77d854244a49c8a19c3ebc547"
format_stamp: "Formatted at 2026-08-08 06:09:17 on dist-test-slave-zj8q"
server_key: "ee4052d18c8b9b17a86c57f41c29b3de"
server_key_iv: "6ec64f19eea3b5873bd0900a2bfbe60d"
server_key_version: "encryptionkey@0"
I20260808 06:09:17.176855 5409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-2/wal/instance:
uuid: "1f50c2e77d854244a49c8a19c3ebc547"
format_stamp: "Formatted at 2026-08-08 06:09:17 on dist-test-slave-zj8q"
server_key: "ee4052d18c8b9b17a86c57f41c29b3de"
server_key_iv: "6ec64f19eea3b5873bd0900a2bfbe60d"
server_key_version: "encryptionkey@0"
I20260808 06:09:17.180613 5409 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260808 06:09:17.182992 5433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 06:09:17.184185 5409 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 06:09:17.184306 5409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-2/data,/tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-2/wal
uuid: "1f50c2e77d854244a49c8a19c3ebc547"
format_stamp: "Formatted at 2026-08-08 06:09:17 on dist-test-slave-zj8q"
server_key: "ee4052d18c8b9b17a86c57f41c29b3de"
server_key_iv: "6ec64f19eea3b5873bd0900a2bfbe60d"
server_key_version: "encryptionkey@0"
I20260808 06:09:17.184396 5409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 06:09:17.199241 5409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 06:09:17.202111 5409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 06:09:17.202324 5409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 06:09:17.209331 5409 rpc_server.cc:307] RPC server started. Bound to: 127.1.151.188:44523
I20260808 06:09:17.209580 5511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.151.188:44523 every 8 connection(s)
I20260808 06:09:17.210312 5409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-2/data/info.pb
I20260808 06:09:17.213049 5512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 06:09:17.216523 1630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu as pid 5409
I20260808 06:09:17.216670 1630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIVYy3F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786169070948234-1630-0/minicluster-data/master-2/wal/instance
I20260808 06:09:17.216981 1630 external_mini_cluster.cc:1428] Setting key c46a78fba6a1b13d82467dde360399f4
I20260808 06:09:17.217007 5512 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.151.190" port: 42471 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 06:09:17 dist-test-slave-zj8q krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169357, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.188@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
Aug 08 06:09:17 dist-test-slave-zj8q krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.188@KRBTEST.COM
I20260808 06:09:17.228950 5294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:17.219083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33616 (local address 127.1.151.190:42471)
0808 06:09:17.219211 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:17.219215 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:17.219482 (+ 267us) server_negotiation.cc:407] Connection header received
0808 06:09:17.219831 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:17.219835 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:17.219891 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:17.219958 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:17.220769 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.221271 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.222243 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.222454 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.222542 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:17.225067 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:17.225071 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:17.225074 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:09:17.225103 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:17.226693 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:17.227253 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:17.227257 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:17.227259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:17.227347 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:17.227680 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:17.227683 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:17.227684 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:17.227933 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:17.228021 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:17.228626 (+ 605us) server_negotiation.cc:299] Negotiation successful
0808 06:09:17.228769 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 06:09:17.229817 5520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:17.218336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53290 (local address 127.1.151.188:44523)
0808 06:09:17.218748 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:17.218765 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:17.218805 (+ 40us) server_negotiation.cc:407] Connection header received
0808 06:09:17.219656 (+ 851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:17.219680 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:17.219879 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:17.220038 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:17.220841 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.221678 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.222527 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.222935 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.223030 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:17.225504 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:17.225509 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:17.225534 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:09:17.225555 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:17.227106 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:17.227545 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:17.227550 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:17.227552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:17.227671 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:17.227914 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:17.227916 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:17.227917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:17.228377 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:17.228583 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:17.228823 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 06:09:17.228940 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":271,"thread_start_us":119,"threads_started":1}
I20260808 06:09:17.229817 5521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:17.219006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.190:42471 (local address 127.0.0.1:33616)
0808 06:09:17.219356 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:17.219395 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 06:09:17.219654 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:09:17.220023 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:09:17.220028 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:09:17.220385 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:09:17.220628 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:09:17.220642 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.221370 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.221373 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:09:17.222105 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:09:17.222114 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.222587 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.222591 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:09:17.222819 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:17.223330 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:09:17.223351 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:09:17.224915 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:09:17.226805 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:09:17.226814 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:09:17.226823 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:09:17.227162 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:09:17.227433 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:09:17.227435 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:09:17.227436 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:09:17.227590 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:09:17.228038 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:09:17.228043 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:09:17.228380 (+ 337us) client_negotiation.cc:769] Sending connection context
0808 06:09:17.228544 (+ 164us) client_negotiation.cc:240] Negotiation successful
0808 06:09:17.228836 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":64,"spinlock_wait_cycles":26880,"thread_start_us":85,"threads_started":1}
I20260808 06:09:17.231297 5512 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.151.189" port: 37567 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 06:09:17 dist-test-slave-zj8q krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169357, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.188@KRBTEST.COM for kudu/127.1.151.189@KRBTEST.COM
Aug 08 06:09:17 dist-test-slave-zj8q krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.190@KRBTEST.COM
I20260808 06:09:17.239508 5521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:17.231751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.189:37567 (local address 127.0.0.1:47138)
0808 06:09:17.231880 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:17.231890 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 06:09:17.231945 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:09:17.232274 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:09:17.232277 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:09:17.232413 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:09:17.232507 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:09:17.232513 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.233289 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.233291 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:09:17.233856 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:09:17.233865 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.234322 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.234326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:09:17.234417 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:17.234873 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:09:17.234887 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:09:17.236176 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:09:17.238024 (+ 1848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:09:17.238028 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:09:17.238030 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:09:17.238292 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:09:17.238571 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:09:17.238573 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:09:17.238575 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:09:17.238654 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:09:17.239037 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:09:17.239040 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:09:17.239136 (+ 96us) client_negotiation.cc:769] Sending connection context
0808 06:09:17.239222 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 06:09:17.239346 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260808 06:09:17.239544 5408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:17.231929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47138 (local address 127.1.151.189:37567)
0808 06:09:17.232057 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:17.232061 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:17.232074 (+ 13us) server_negotiation.cc:407] Connection header received
0808 06:09:17.232117 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:17.232120 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:17.232165 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:17.232233 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:17.232631 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.233186 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.233972 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.234232 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.234305 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:17.236310 (+ 2005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:17.236313 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:17.236315 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:09:17.236362 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:17.237893 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:17.238383 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:17.238386 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:17.238388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:17.238483 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:17.238734 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:17.238737 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:17.238739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:17.238944 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:17.239028 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:17.239289 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 06:09:17.239399 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 06:09:17.240296 5294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:17.232077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33622 (local address 127.1.151.190:42471)
0808 06:09:17.232360 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:17.232363 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:17.232372 (+ 9us) server_negotiation.cc:407] Connection header received
0808 06:09:17.232397 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:17.232399 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:17.232429 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:17.232481 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:17.233073 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.233583 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.234452 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.234690 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.234784 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:17.237193 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:17.237196 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:17.237198 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:09:17.237223 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:17.238753 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:17.239138 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:17.239142 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:17.239143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:17.239216 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:17.239452 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:17.239455 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:17.239457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:17.239692 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:17.239911 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:17.240045 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 06:09:17.240148 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220}
I20260808 06:09:17.240619 5512 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44523 } has no permanent_uuid. Determining permanent_uuid...
I20260808 06:09:17.245149 5401 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44523 } attempt: 3
Aug 08 06:09:17 dist-test-slave-zj8q krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169357, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.188@KRBTEST.COM for kudu/127.1.151.188@KRBTEST.COM
Aug 08 06:09:17 dist-test-slave-zj8q krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.151.189@KRBTEST.COM
I20260808 06:09:17.248956 5520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:17.241111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53304 (local address 127.1.151.188:44523)
0808 06:09:17.241214 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:17.241218 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:17.241361 (+ 143us) server_negotiation.cc:407] Connection header received
0808 06:09:17.241386 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:17.241388 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:17.241422 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:17.241489 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:17.241895 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.242530 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.243392 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.243611 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.243706 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:17.245729 (+ 2023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:17.245733 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:17.245735 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:09:17.245757 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:17.247467 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:17.247895 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:17.247898 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:17.247900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:17.247971 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:17.248221 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:17.248224 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:17.248226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:17.248376 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:17.248455 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:17.248672 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 06:09:17.248806 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":32}
I20260808 06:09:17.248942 5521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:17.241066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:44523 (local address 127.0.0.1:53304)
0808 06:09:17.241212 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:17.241224 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 06:09:17.241297 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:09:17.241525 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:09:17.241528 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:09:17.241697 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:09:17.241785 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:09:17.241792 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.242648 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.242650 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:09:17.243280 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:09:17.243290 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.243715 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.243719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:09:17.243786 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:17.244151 (+ 365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:09:17.244167 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:09:17.245586 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:09:17.247563 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:09:17.247566 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:09:17.247568 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:09:17.247797 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:09:17.248065 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:09:17.248068 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:09:17.248070 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:09:17.248144 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:09:17.248457 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:09:17.248460 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:09:17.248536 (+ 76us) client_negotiation.cc:769] Sending connection context
0808 06:09:17.248617 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 06:09:17.248753 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97}
Aug 08 06:09:17 dist-test-slave-zj8q krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169356, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.189@KRBTEST.COM for kudu/127.1.151.188@KRBTEST.COM
I20260808 06:09:17.250977 5408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:17.241325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47144 (local address 127.1.151.189:37567)
0808 06:09:17.241429 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:17.241433 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:17.241444 (+ 11us) server_negotiation.cc:407] Connection header received
0808 06:09:17.241510 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:17.241512 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:17.241549 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:17.241613 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:17.242246 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.242856 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.243712 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.243954 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.244041 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:17.246664 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:17.246669 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:17.246672 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:09:17.246698 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:17.249366 (+ 2668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:17.249833 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:17.249837 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:17.249838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:17.249918 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:17.250202 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:17.250206 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:17.250208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:17.250375 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:17.250592 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:17.250737 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 06:09:17.250835 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 06:09:17.252961 5512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f50c2e77d854244a49c8a19c3ebc547: Bootstrap starting.
I20260808 06:09:17.253187 5288 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44523 } attempt: 3
I20260808 06:09:17.254407 5406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:17.245498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:44523 (local address 127.0.0.1:53306)
0808 06:09:17.245673 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:17.245688 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 06:09:17.245759 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:09:17.246261 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:09:17.246265 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:09:17.246425 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:09:17.246513 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:09:17.246519 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.247919 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.247923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:09:17.248419 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:09:17.248427 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.249035 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.249039 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:09:17.249129 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:17.249533 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:09:17.249547 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:09:17.250942 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:09:17.252740 (+ 1798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:09:17.252744 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:09:17.252746 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:09:17.252971 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:09:17.253297 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:09:17.253300 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:09:17.253302 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:09:17.253402 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:09:17.253918 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:09:17.253921 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:09:17.254023 (+ 102us) client_negotiation.cc:769] Sending connection context
0808 06:09:17.254102 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 06:09:17.254265 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115}
I20260808 06:09:17.254823 1630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 06:09:17.255293 5522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:17.245576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53306 (local address 127.1.151.188:44523)
0808 06:09:17.245960 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:17.245966 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:17.245977 (+ 11us) server_negotiation.cc:407] Connection header received
0808 06:09:17.246011 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:17.246013 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:17.246051 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:17.246242 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:17.246619 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.247827 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.248525 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.248883 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.249026 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:17.251077 (+ 2051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:17.251080 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:17.251083 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:09:17.251105 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:17.252634 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:17.253066 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:17.253069 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:17.253071 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:17.253186 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:17.253546 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:17.253550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:17.253551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:17.253809 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:17.254753 (+ 944us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:17.254828 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 06:09:17.255109 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":113,"threads_started":1}
I20260808 06:09:17.256584 5512 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f50c2e77d854244a49c8a19c3ebc547: Neither blocks nor log segments found. Creating new log.
I20260808 06:09:17.257534 5512 log.cc:824] T 00000000000000000000000000000000 P 1f50c2e77d854244a49c8a19c3ebc547: Log is configured to *not* fsync() on all Append() calls
I20260808 06:09:17.256181 5242 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "81217163bd8e4a0c9a4408aa2656bcee"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33622
Aug 08 06:09:17 dist-test-slave-zj8q krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786169356, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.151.190@KRBTEST.COM for kudu/127.1.151.188@KRBTEST.COM
I20260808 06:09:17.258177 5401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c3ac46dd7104dc89dfa7a82e793fec1: Bootstrap starting.
I20260808 06:09:17.259927 5512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f50c2e77d854244a49c8a19c3ebc547: No bootstrap required, opened a new log
I20260808 06:09:17.260667 5401 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c3ac46dd7104dc89dfa7a82e793fec1: Neither blocks nor log segments found. Creating new log.
I20260808 06:09:17.261489 5401 log.cc:824] T 00000000000000000000000000000000 P 9c3ac46dd7104dc89dfa7a82e793fec1: Log is configured to *not* fsync() on all Append() calls
I20260808 06:09:17.261786 5520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:17.253695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53314 (local address 127.1.151.188:44523)
0808 06:09:17.253791 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:17.253795 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:17.253806 (+ 11us) server_negotiation.cc:407] Connection header received
0808 06:09:17.253835 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:17.253838 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:17.253874 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:17.254435 (+ 561us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:17.254520 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.255178 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.256196 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.256490 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.256913 (+ 423us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:17.258709 (+ 1796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:17.258712 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:17.258715 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:09:17.258738 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:17.260227 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:17.260624 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:17.260627 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:17.260629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:17.260711 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:17.261014 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:17.261016 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:17.261018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:17.261214 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:17.261299 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:17.261543 (+ 244us) server_negotiation.cc:299] Negotiation successful
0808 06:09:17.261641 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 06:09:17.262105 5296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:17.253537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.151.188:44523 (local address 127.0.0.1:53314)
0808 06:09:17.253645 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 06:09:17.253659 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 06:09:17.253734 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 06:09:17.253993 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 06:09:17.253996 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 06:09:17.254210 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0808 06:09:17.254323 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:09:17.254332 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.255308 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.255312 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:09:17.256045 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 06:09:17.256055 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.256615 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.256619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 06:09:17.256704 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:17.257111 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 06:09:17.257125 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 06:09:17.258566 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 06:09:17.260328 (+ 1762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:09:17.260331 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:09:17.260334 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:09:17.260532 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:09:17.260816 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 06:09:17.260819 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 06:09:17.260821 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 06:09:17.260909 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 06:09:17.261303 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 06:09:17.261306 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 06:09:17.261395 (+ 89us) client_negotiation.cc:769] Sending connection context
0808 06:09:17.261479 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 06:09:17.261735 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 06:09:17.263530 5401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c3ac46dd7104dc89dfa7a82e793fec1: No bootstrap required, opened a new log
I20260808 06:09:17.264183 5512 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f50c2e77d854244a49c8a19c3ebc547 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81217163bd8e4a0c9a4408aa2656bcee" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 42471 } } peers { permanent_uuid: "9c3ac46dd7104dc89dfa7a82e793fec1" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 37567 } } peers { permanent_uuid: "1f50c2e77d854244a49c8a19c3ebc547" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44523 } }
I20260808 06:09:17.264416 5512 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f50c2e77d854244a49c8a19c3ebc547 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f50c2e77d854244a49c8a19c3ebc547, State: Initialized, Role: FOLLOWER
I20260808 06:09:17.264737 5512 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f50c2e77d854244a49c8a19c3ebc547 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81217163bd8e4a0c9a4408aa2656bcee" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 42471 } } peers { permanent_uuid: "9c3ac46dd7104dc89dfa7a82e793fec1" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 37567 } } peers { permanent_uuid: "1f50c2e77d854244a49c8a19c3ebc547" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44523 } }
I20260808 06:09:17.265007 5288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81217163bd8e4a0c9a4408aa2656bcee: Bootstrap starting.
I20260808 06:09:17.265324 5528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f50c2e77d854244a49c8a19c3ebc547 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81217163bd8e4a0c9a4408aa2656bcee" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 42471 } } peers { permanent_uuid: "9c3ac46dd7104dc89dfa7a82e793fec1" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 37567 } } peers { permanent_uuid: "1f50c2e77d854244a49c8a19c3ebc547" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44523 } } }
I20260808 06:09:17.265568 5528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f50c2e77d854244a49c8a19c3ebc547 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 06:09:17.266090 5512 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f50c2e77d854244a49c8a19c3ebc547 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:09:17.267371 5401 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c3ac46dd7104dc89dfa7a82e793fec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81217163bd8e4a0c9a4408aa2656bcee" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 42471 } } peers { permanent_uuid: "9c3ac46dd7104dc89dfa7a82e793fec1" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 37567 } } peers { permanent_uuid: "1f50c2e77d854244a49c8a19c3ebc547" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44523 } }
I20260808 06:09:17.267546 5288 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81217163bd8e4a0c9a4408aa2656bcee: Neither blocks nor log segments found. Creating new log.
I20260808 06:09:17.267561 5401 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c3ac46dd7104dc89dfa7a82e793fec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c3ac46dd7104dc89dfa7a82e793fec1, State: Initialized, Role: FOLLOWER
I20260808 06:09:17.267863 5401 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c3ac46dd7104dc89dfa7a82e793fec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81217163bd8e4a0c9a4408aa2656bcee" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 42471 } } peers { permanent_uuid: "9c3ac46dd7104dc89dfa7a82e793fec1" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 37567 } } peers { permanent_uuid: "1f50c2e77d854244a49c8a19c3ebc547" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44523 } }
I20260808 06:09:17.268342 5288 log.cc:824] T 00000000000000000000000000000000 P 81217163bd8e4a0c9a4408aa2656bcee: Log is configured to *not* fsync() on all Append() calls
I20260808 06:09:17.268409 5544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c3ac46dd7104dc89dfa7a82e793fec1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81217163bd8e4a0c9a4408aa2656bcee" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 42471 } } peers { permanent_uuid: "9c3ac46dd7104dc89dfa7a82e793fec1" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 37567 } } peers { permanent_uuid: "1f50c2e77d854244a49c8a19c3ebc547" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44523 } } }
I20260808 06:09:17.268657 5544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c3ac46dd7104dc89dfa7a82e793fec1 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 06:09:17.269122 5401 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c3ac46dd7104dc89dfa7a82e793fec1 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 06:09:17.271926 5549 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1f50c2e77d854244a49c8a19c3ebc547: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 06:09:17.272009 5549 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 06:09:17.270371 5409 master_runner.cc:186] Error getting master registration for 127.1.151.190:42471: OK, instance_id { permanent_uuid: "81217163bd8e4a0c9a4408aa2656bcee" instance_seqno: 1786169356792031 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 06:09:17.273551 5409 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 06:09:17.273746 5288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81217163bd8e4a0c9a4408aa2656bcee: No bootstrap required, opened a new log
I20260808 06:09:17.272565 5300 master_runner.cc:186] Error getting master registration for 127.1.151.190:42471: OK, instance_id { permanent_uuid: "81217163bd8e4a0c9a4408aa2656bcee" instance_seqno: 1786169356792031 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 06:09:17.276304 5300 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 06:09:17.276767 5288 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81217163bd8e4a0c9a4408aa2656bcee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81217163bd8e4a0c9a4408aa2656bcee" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 42471 } } peers { permanent_uuid: "9c3ac46dd7104dc89dfa7a82e793fec1" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 37567 } } peers { permanent_uuid: "1f50c2e77d854244a49c8a19c3ebc547" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44523 } }
I20260808 06:09:17.276971 5288 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81217163bd8e4a0c9a4408aa2656bcee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81217163bd8e4a0c9a4408aa2656bcee, State: Initialized, Role: FOLLOWER
I20260808 06:09:17.277406 5288 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81217163bd8e4a0c9a4408aa2656bcee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81217163bd8e4a0c9a4408aa2656bcee" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 42471 } } peers { permanent_uuid: "9c3ac46dd7104dc89dfa7a82e793fec1" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 37567 } } peers { permanent_uuid: "1f50c2e77d854244a49c8a19c3ebc547" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44523 } }
I20260808 06:09:17.277966 5564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81217163bd8e4a0c9a4408aa2656bcee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81217163bd8e4a0c9a4408aa2656bcee" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 42471 } } peers { permanent_uuid: "9c3ac46dd7104dc89dfa7a82e793fec1" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 37567 } } peers { permanent_uuid: "1f50c2e77d854244a49c8a19c3ebc547" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44523 } } }
I20260808 06:09:17.278223 5564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81217163bd8e4a0c9a4408aa2656bcee [sys.catalog]: This master's current role is: FOLLOWER
I20260808 06:09:17.278816 5288 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81217163bd8e4a0c9a4408aa2656bcee [sys.catalog]: configured and running, proceeding with master startup.
I20260808 06:09:17.278858 5242 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81217163bd8e4a0c9a4408aa2656bcee [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 06:09:17.278970 5242 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81217163bd8e4a0c9a4408aa2656bcee [term 0 FOLLOWER]: Advancing to term 1
W20260808 06:09:17.279568 5563 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9c3ac46dd7104dc89dfa7a82e793fec1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 06:09:17.279626 5563 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 06:09:17.280176 5242 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81217163bd8e4a0c9a4408aa2656bcee [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81217163bd8e4a0c9a4408aa2656bcee" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 42471 } } peers { permanent_uuid: "9c3ac46dd7104dc89dfa7a82e793fec1" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 37567 } } peers { permanent_uuid: "1f50c2e77d854244a49c8a19c3ebc547" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44523 } }
I20260808 06:09:17.281100 5242 leader_election.cc:288] T 00000000000000000000000000000000 P 81217163bd8e4a0c9a4408aa2656bcee [CANDIDATE]: Term 1 election: Requested vote from peers 9c3ac46dd7104dc89dfa7a82e793fec1 (127.1.151.189:37567), 1f50c2e77d854244a49c8a19c3ebc547 (127.1.151.188:44523)
I20260808 06:09:17.281673 5477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "81217163bd8e4a0c9a4408aa2656bcee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1f50c2e77d854244a49c8a19c3ebc547"
I20260808 06:09:17.281890 5477 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f50c2e77d854244a49c8a19c3ebc547 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:09:17.282142 5357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "81217163bd8e4a0c9a4408aa2656bcee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9c3ac46dd7104dc89dfa7a82e793fec1"
I20260808 06:09:17.282366 5357 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c3ac46dd7104dc89dfa7a82e793fec1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 06:09:17.283231 5477 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1f50c2e77d854244a49c8a19c3ebc547 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81217163bd8e4a0c9a4408aa2656bcee in term 1.
I20260808 06:09:17.283663 5357 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9c3ac46dd7104dc89dfa7a82e793fec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81217163bd8e4a0c9a4408aa2656bcee in term 1.
W20260808 06:09:17.284752 5593 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 81217163bd8e4a0c9a4408aa2656bcee: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 06:09:17.284827 5593 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 06:09:17.284785 5213 leader_election.cc:302] T 00000000000000000000000000000000 P 81217163bd8e4a0c9a4408aa2656bcee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81217163bd8e4a0c9a4408aa2656bcee, 9c3ac46dd7104dc89dfa7a82e793fec1; no voters:
I20260808 06:09:17.285192 5564 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81217163bd8e4a0c9a4408aa2656bcee [term 1 FOLLOWER]: Leader election won for term 1
I20260808 06:09:17.285390 5564 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81217163bd8e4a0c9a4408aa2656bcee [term 1 LEADER]: Becoming Leader. State: Replica: 81217163bd8e4a0c9a4408aa2656bcee, State: Running, Role: LEADER
I20260808 06:09:17.285727 5564 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81217163bd8e4a0c9a4408aa2656bcee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81217163bd8e4a0c9a4408aa2656bcee" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 42471 } } peers { permanent_uuid: "9c3ac46dd7104dc89dfa7a82e793fec1" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 37567 } } peers { permanent_uuid: "1f50c2e77d854244a49c8a19c3ebc547" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44523 } }
I20260808 06:09:17.287523 5564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81217163bd8e4a0c9a4408aa2656bcee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81217163bd8e4a0c9a4408aa2656bcee. Latest consensus state: current_term: 1 leader_uuid: "81217163bd8e4a0c9a4408aa2656bcee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81217163bd8e4a0c9a4408aa2656bcee" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 42471 } } peers { permanent_uuid: "9c3ac46dd7104dc89dfa7a82e793fec1" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 37567 } } peers { permanent_uuid: "1f50c2e77d854244a49c8a19c3ebc547" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44523 } } }
I20260808 06:09:17.287621 5564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81217163bd8e4a0c9a4408aa2656bcee [sys.catalog]: This master's current role is: LEADER
I20260808 06:09:17.288287 5605 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260808 06:09:17.288570 5217 tablet.cc:2369] T 00000000000000000000000000000000 P 81217163bd8e4a0c9a4408aa2656bcee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 06:09:17.288798 5187 master_runner.cc:263] No leader master found from master 81217163bd8e4a0c9a4408aa2656bcee
I20260808 06:09:17.288857 5187 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 06:09:17.289881 5605 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 06:09:17.297341 5477 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1f50c2e77d854244a49c8a19c3ebc547 [term 1 FOLLOWER]: Refusing update from remote peer 81217163bd8e4a0c9a4408aa2656bcee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:09:17.297341 5357 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9c3ac46dd7104dc89dfa7a82e793fec1 [term 1 FOLLOWER]: Refusing update from remote peer 81217163bd8e4a0c9a4408aa2656bcee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 06:09:17.298091 5564 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 81217163bd8e4a0c9a4408aa2656bcee [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c3ac46dd7104dc89dfa7a82e793fec1" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 37567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:09:17.298341 5599 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 81217163bd8e4a0c9a4408aa2656bcee [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f50c2e77d854244a49c8a19c3ebc547" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 06:09:17.305245 5544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c3ac46dd7104dc89dfa7a82e793fec1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81217163bd8e4a0c9a4408aa2656bcee. Latest consensus state: current_term: 1 leader_uuid: "81217163bd8e4a0c9a4408aa2656bcee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81217163bd8e4a0c9a4408aa2656bcee" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 42471 } } peers { permanent_uuid: "9c3ac46dd7104dc89dfa7a82e793fec1" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 37567 } } peers { permanent_uuid: "1f50c2e77d854244a49c8a19c3ebc547" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44523 } } }
I20260808 06:09:17.305383 5544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c3ac46dd7104dc89dfa7a82e793fec1 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 06:09:17.306927 5528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f50c2e77d854244a49c8a19c3ebc547 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81217163bd8e4a0c9a4408aa2656bcee. Latest consensus state: current_term: 1 leader_uuid: "81217163bd8e4a0c9a4408aa2656bcee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81217163bd8e4a0c9a4408aa2656bcee" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 42471 } } peers { permanent_uuid: "9c3ac46dd7104dc89dfa7a82e793fec1" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 37567 } } peers { permanent_uuid: "1f50c2e77d854244a49c8a19c3ebc547" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44523 } } }
I20260808 06:09:17.307070 5528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f50c2e77d854244a49c8a19c3ebc547 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 06:09:17.308276 5408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:17.286306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47154 (local address 127.1.151.189:37567)
0808 06:09:17.286407 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:17.286412 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:17.298613 (+ 12201us) server_negotiation.cc:407] Connection header received
0808 06:09:17.298654 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:17.298656 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:17.298699 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:17.298761 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:17.299725 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.300517 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.301694 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.302027 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.302113 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:17.303941 (+ 1828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:17.303944 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:17.303947 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:09:17.303974 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:17.306138 (+ 2164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:17.306605 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:17.306609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:17.306611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:17.306746 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:17.307041 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:17.307045 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:17.307047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:17.307264 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:17.307383 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:17.307995 (+ 612us) server_negotiation.cc:299] Negotiation successful
0808 06:09:17.308111 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 06:09:17.316712 5520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:17.299396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53318 (local address 127.1.151.188:44523)
0808 06:09:17.299507 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:17.299511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:17.305486 (+ 5975us) server_negotiation.cc:407] Connection header received
0808 06:09:17.305748 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:17.305751 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:17.305804 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:17.305854 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:17.309313 (+ 3459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.310080 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.311125 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.311435 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.311509 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:17.313223 (+ 1714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:17.313228 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:17.313230 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:09:17.313258 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:17.314846 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:17.315315 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:17.315319 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:17.315321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:17.315406 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:17.315774 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:17.315778 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:17.315779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:17.315995 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:17.316101 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:17.316424 (+ 323us) server_negotiation.cc:299] Negotiation successful
0808 06:09:17.316539 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 06:09:17.317200 5294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:17.285932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33630 (local address 127.1.151.190:42471)
0808 06:09:17.286042 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:17.286047 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:17.298472 (+ 12425us) server_negotiation.cc:407] Connection header received
0808 06:09:17.298630 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:17.298633 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:17.298695 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:17.298747 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:17.303292 (+ 4545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.303994 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.310246 (+ 6252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:17.310604 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:17.310707 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:17.313681 (+ 2974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 06:09:17.313691 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 06:09:17.313694 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 06:09:17.313726 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 06:09:17.315407 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:17.315862 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:17.315866 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:17.315867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:17.315946 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 06:09:17.316258 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 06:09:17.316262 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 06:09:17.316263 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 06:09:17.316486 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 06:09:17.316589 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:17.316926 (+ 337us) server_negotiation.cc:299] Negotiation successful
0808 06:09:17.317040 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 06:09:17.321000 5599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81217163bd8e4a0c9a4408aa2656bcee [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "81217163bd8e4a0c9a4408aa2656bcee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81217163bd8e4a0c9a4408aa2656bcee" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 42471 } } peers { permanent_uuid: "9c3ac46dd7104dc89dfa7a82e793fec1" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 37567 } } peers { permanent_uuid: "1f50c2e77d854244a49c8a19c3ebc547" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44523 } } }
I20260808 06:09:17.321132 5599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81217163bd8e4a0c9a4408aa2656bcee [sys.catalog]: This master's current role is: LEADER
I20260808 06:09:17.321250 5606 mvcc.cc:205] Tried to move back new op lower bound from 7316149687480266752 to 7316149687445790720. Current Snapshot: MvccSnapshot[applied={T|T < 7316149687480266752}]
I20260808 06:09:17.322557 5544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c3ac46dd7104dc89dfa7a82e793fec1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "81217163bd8e4a0c9a4408aa2656bcee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81217163bd8e4a0c9a4408aa2656bcee" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 42471 } } peers { permanent_uuid: "9c3ac46dd7104dc89dfa7a82e793fec1" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 37567 } } peers { permanent_uuid: "1f50c2e77d854244a49c8a19c3ebc547" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44523 } } }
I20260808 06:09:17.322691 5544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c3ac46dd7104dc89dfa7a82e793fec1 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 06:09:17.324879 5614 mvcc.cc:205] Tried to move back new op lower bound from 7316149687480266752 to 7316149687445790720. Current Snapshot: MvccSnapshot[applied={T|T < 7316149687480266752}]
I20260808 06:09:17.328152 5605 catalog_manager.cc:1347] Generated new cluster ID: ec6fac3f5d434233bc8b4e6462b5b7ab
I20260808 06:09:17.328222 5605 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 06:09:17.329258 5564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81217163bd8e4a0c9a4408aa2656bcee [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "81217163bd8e4a0c9a4408aa2656bcee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81217163bd8e4a0c9a4408aa2656bcee" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 42471 } } peers { permanent_uuid: "9c3ac46dd7104dc89dfa7a82e793fec1" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 37567 } } peers { permanent_uuid: "1f50c2e77d854244a49c8a19c3ebc547" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44523 } } }
I20260808 06:09:17.329375 5564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81217163bd8e4a0c9a4408aa2656bcee [sys.catalog]: This master's current role is: LEADER
I20260808 06:09:17.330461 5528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f50c2e77d854244a49c8a19c3ebc547 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "81217163bd8e4a0c9a4408aa2656bcee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81217163bd8e4a0c9a4408aa2656bcee" member_type: VOTER last_known_addr { host: "127.1.151.190" port: 42471 } } peers { permanent_uuid: "9c3ac46dd7104dc89dfa7a82e793fec1" member_type: VOTER last_known_addr { host: "127.1.151.189" port: 37567 } } peers { permanent_uuid: "1f50c2e77d854244a49c8a19c3ebc547" member_type: VOTER last_known_addr { host: "127.1.151.188" port: 44523 } } }
I20260808 06:09:17.330577 5528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f50c2e77d854244a49c8a19c3ebc547 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 06:09:17.366679 5605 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 06:09:17.370290 5605 catalog_manager.cc:1504] Loading token signing keys...
I20260808 06:09:17.393325 5605 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81217163bd8e4a0c9a4408aa2656bcee: Generated new TSK 0
I20260808 06:09:17.394294 5605 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 06:09:17.448249 5408 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:09:17.446918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47170 (local address 127.1.151.189:37567)
0808 06:09:17.447041 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:17.447046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:17.447495 (+ 449us) server_negotiation.cc:407] Connection header received
0808 06:09:17.447614 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:17.447617 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:17.447672 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:17.447765 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:17.448113 (+ 348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47170: BlockingRecv error: recv got EOF from 127.0.0.1:47170 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260808 06:09:17.448249 5294 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:09:17.441150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33638 (local address 127.1.151.190:42471)
0808 06:09:17.441284 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:17.441289 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:17.447005 (+ 5716us) server_negotiation.cc:407] Connection header received
0808 06:09:17.447190 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:17.447194 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:17.447256 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:17.447334 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:17.448113 (+ 779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33638: BlockingRecv error: recv got EOF from 127.0.0.1:33638 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260808 06:09:17.449404 5520 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:09:17.446587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53326 (local address 127.1.151.188:44523)
0808 06:09:17.446728 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:17.446733 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:17.447533 (+ 800us) server_negotiation.cc:407] Connection header received
0808 06:09:17.448572 (+ 1039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:17.448575 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:17.448626 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:17.448680 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:17.449271 (+ 591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53326: BlockingRecv error: recv got EOF from 127.0.0.1:53326 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 06:09:17.456032 5408 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:09:17.452016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47174 (local address 127.1.151.189:37567)
0808 06:09:17.452121 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:17.452126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:17.454977 (+ 2851us) server_negotiation.cc:407] Connection header received
0808 06:09:17.455225 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:17.455229 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:17.455312 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:17.455371 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:17.455927 (+ 556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47174: BlockingRecv error: recv got EOF from 127.0.0.1:47174 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 06:09:17.465021 5408 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:09:17.463469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47188 (local address 127.1.151.189:37567)
0808 06:09:17.463594 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:17.463599 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:17.464107 (+ 508us) server_negotiation.cc:407] Connection header received
0808 06:09:17.464300 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:17.464304 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:17.464356 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:17.464405 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:17.464921 (+ 516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47188: BlockingRecv error: recv got EOF from 127.0.0.1:47188 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260808 06:09:17.470402 5408 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:09:17.469171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47190 (local address 127.1.151.189:37567)
0808 06:09:17.469290 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:17.469293 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:17.469564 (+ 271us) server_negotiation.cc:407] Connection header received
0808 06:09:17.469733 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:17.469737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:17.469782 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:17.469827 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:17.470307 (+ 480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47190: BlockingRecv error: recv got EOF from 127.0.0.1:47190 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260808 06:09:17.488673 5408 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:09:17.476392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47202 (local address 127.1.151.189:37567)
0808 06:09:17.476559 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:17.476564 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:17.487690 (+ 11126us) server_negotiation.cc:407] Connection header received
0808 06:09:17.487882 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:17.487885 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:17.487937 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:17.487987 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:17.488522 (+ 535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47202: BlockingRecv error: recv got EOF from 127.0.0.1:47202 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260808 06:09:17.499765 5408 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:09:17.498594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47204 (local address 127.1.151.189:37567)
0808 06:09:17.498698 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:17.498701 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:17.498994 (+ 293us) server_negotiation.cc:407] Connection header received
0808 06:09:17.499127 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:17.499131 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:17.499178 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:17.499242 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:17.499657 (+ 415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47204: BlockingRecv error: recv got EOF from 127.0.0.1:47204 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260808 06:09:17.518775 5408 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:09:17.517507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47216 (local address 127.1.151.189:37567)
0808 06:09:17.517622 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:17.517626 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:17.517875 (+ 249us) server_negotiation.cc:407] Connection header received
0808 06:09:17.518019 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:17.518023 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:17.518078 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:17.518197 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:17.518656 (+ 459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47216: BlockingRecv error: recv got EOF from 127.0.0.1:47216 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 06:09:17.553995 5408 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:09:17.552984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47220 (local address 127.1.151.189:37567)
0808 06:09:17.553167 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:17.553172 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:17.553244 (+ 72us) server_negotiation.cc:407] Connection header received
0808 06:09:17.553370 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:17.553373 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:17.553421 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:17.553491 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:17.553873 (+ 382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47220: BlockingRecv error: recv got EOF from 127.0.0.1:47220 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260808 06:09:17.621344 5408 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:09:17.620185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47236 (local address 127.1.151.189:37567)
0808 06:09:17.620338 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:17.620343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:17.620440 (+ 97us) server_negotiation.cc:407] Connection header received
0808 06:09:17.620600 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:17.620604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:17.620661 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:17.620734 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:17.621219 (+ 485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47236: BlockingRecv error: recv got EOF from 127.0.0.1:47236 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260808 06:09:17.752710 5408 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:09:17.751425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47240 (local address 127.1.151.189:37567)
0808 06:09:17.751613 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:17.751619 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:17.751708 (+ 89us) server_negotiation.cc:407] Connection header received
0808 06:09:17.751872 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:17.751875 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:17.751923 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:17.751994 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:17.752595 (+ 601us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47240: BlockingRecv error: recv got EOF from 127.0.0.1:47240 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260808 06:09:18.011749 5408 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 06:09:18.010749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47254 (local address 127.1.151.189:37567)
0808 06:09:18.010914 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:18.010918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:18.010971 (+ 53us) server_negotiation.cc:407] Connection header received
0808 06:09:18.011110 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:18.011113 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:18.011162 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:18.011224 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 06:09:18.011636 (+ 412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47254: BlockingRecv error: recv got EOF from 127.0.0.1:47254 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 06:09:18.273352 5549 catalog_manager.cc:1259] Loaded cluster ID: ec6fac3f5d434233bc8b4e6462b5b7ab
I20260808 06:09:18.273437 5549 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1f50c2e77d854244a49c8a19c3ebc547: loading cluster ID for follower catalog manager: success
I20260808 06:09:18.275213 5549 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1f50c2e77d854244a49c8a19c3ebc547: acquiring CA information for follower catalog manager: success
I20260808 06:09:18.276373 5549 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1f50c2e77d854244a49c8a19c3ebc547: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 06:09:18.280714 5563 catalog_manager.cc:1259] Loaded cluster ID: ec6fac3f5d434233bc8b4e6462b5b7ab
I20260808 06:09:18.280772 5563 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9c3ac46dd7104dc89dfa7a82e793fec1: loading cluster ID for follower catalog manager: success
I20260808 06:09:18.282400 5563 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9c3ac46dd7104dc89dfa7a82e793fec1: acquiring CA information for follower catalog manager: success
I20260808 06:09:18.283397 5563 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9c3ac46dd7104dc89dfa7a82e793fec1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 06:09:18.529835 5738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:18.525899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47256 (local address 127.1.151.189:37567)
0808 06:09:18.526206 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:18.526211 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:18.526228 (+ 17us) server_negotiation.cc:407] Connection header received
0808 06:09:18.526380 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:18.526384 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:18.526516 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:18.526614 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:09:18.526902 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:18.527444 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:18.528308 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:18.528545 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:18.528635 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:18.528976 (+ 341us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:09:18.529425 (+ 449us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:09:18.529537 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:18.529612 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 06:09:18.529677 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":69,"threads_started":1}
I20260808 06:09:18.536016 5745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:18.532627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53330 (local address 127.1.151.188:44523)
0808 06:09:18.532867 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:18.532872 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:18.532886 (+ 14us) server_negotiation.cc:407] Connection header received
0808 06:09:18.532968 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:18.532970 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:18.533077 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:18.533177 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:09:18.533364 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:18.533881 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:18.534704 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:18.534952 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:18.535068 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:18.535278 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:09:18.535683 (+ 405us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:09:18.535777 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:18.535804 (+ 27us) server_negotiation.cc:299] Negotiation successful
0808 06:09:18.535855 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":74,"threads_started":1}
I20260808 06:09:18.543064 5738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:18.538684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47268 (local address 127.1.151.189:37567)
0808 06:09:18.538798 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:18.538802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:18.539218 (+ 416us) server_negotiation.cc:407] Connection header received
0808 06:09:18.539339 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:18.539343 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:18.539394 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:18.539466 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:09:18.539749 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:18.540554 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:18.541976 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:18.542267 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:18.542351 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:18.542595 (+ 244us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:09:18.542665 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:09:18.542727 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:18.542876 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 06:09:18.542935 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 06:09:18.543089 5757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:18.538714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33650 (local address 127.1.151.190:42471)
0808 06:09:18.539157 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:18.539162 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:18.539399 (+ 237us) server_negotiation.cc:407] Connection header received
0808 06:09:18.539647 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:18.539651 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:18.539812 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:18.539922 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:09:18.540118 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:18.540677 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:18.541535 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:18.541765 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:18.541850 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:18.542185 (+ 335us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:09:18.542502 (+ 317us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:09:18.542588 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:18.542876 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 06:09:18.542938 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":64,"threads_started":1}
I20260808 06:09:18.543212 5745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 06:09:18.539068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53342 (local address 127.1.151.188:44523)
0808 06:09:18.539163 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 06:09:18.539167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 06:09:18.539226 (+ 59us) server_negotiation.cc:407] Connection header received
0808 06:09:18.539410 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 06:09:18.539412 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 06:09:18.539461 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 06:09:18.539524 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 06:09:18.539831 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:18.540580 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:18.541964 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 06:09:18.542292 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 06:09:18.542379 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 06:09:18.542760 (+ 381us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 06:09:18.542838 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 06:09:18.542897 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 06:09:18.543039 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 06:09:18.543089 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 06:09:18.546046 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 5187
I20260808 06:09:18.561409 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 5300
I20260808 06:09:18.573499 1630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIVYy3F/build/debug/bin/kudu with pid 5409
2026-08-08T06:09:18Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3228 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6429 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (287624 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST